vBrownBag
vBrownBag
vBrownBag
Getting Started with Local AI (2/3)
48 minutes Posted Jul 6, 2026 at 9:17 pm.
Welcome & Introduction
Du'An's New Role at Akamai Cloud
Data Privacy and the Case for Self-Hosted AI
Anthropic and OpenAI as the New Cloud Layer
Local Models for Specific Use Cases Cancer Detection Example
GPU Memory Math Weights, KV Cache, and Context Windows
Continuous Batching and GPU Time Slicing
Observability with Langfuse Live Demo
Agent Architectures Single Agent, Workflow, Graph, Swarm, Supervisor
Token Economics, Prompt Caching, and GPU Cost Planning
Ollama vs vLLM Prototyping vs Production
0:00
48:34
Download MP3
Show notes
Join us as Du'An digs into the real mechanics of running AI locally and in production - from GPU memory math to multi-agent architectures, observability, and the economics of self-hosted inference.
Du'An walks through how model weights and KV cache compete for GPU memory, why continuous batching matters when you have more than a handful of users, and how agent architectures like single-agent, workflow, graph, swarm, and supervisor patterns each solve different problems. You will learn how to instrument your agents with Langfuse for observability and cost tracking, when to use Ollama versus vLLM, how prompt caching can cut provider costs by up to 75%, and why GPUs should never sit idle. Episode two of three - the next episode covers deploying at scale.
Timestamps
How to find Du'An:
https://duanlightfoot.com
https://www.linkedin.com/in/duanlightfoot/
Links from the show:
https://langfuse.com/
https://github.com/akamai-developers/akamai-workshop-solution-architect-agent
https://amzn.to/4bvHn1p
https://vllm.ai/