The Infra Pod
The Infra Pod
The Infra Pod
Betting on Open Source Models to be the future (Chat with Benny, Cofounder of Fireworks AI)
40 minutes Posted Apr 28, 2026 at 4:00 pm.
Benny's journey and founding Fireworks AI3:23 Early conviction: betting on inference before ChatGPT8:29 Pivoting from PyTorch training to text inference15:42 Horizontal vs. per-model optimization strategies11:14 Open source vs. frontier models: the real gap32:35 How customers engage: PLG to hands-on customization17:37 When to move off frontier models33:42 The future of agentic memory and data sovereignty32:35 Fireworks' differentiation in a crowded market33:53 Spicy Future: AI doomers, bot management, and going fully out of loop
0:00
40:34
Download MP3
Show notes
In this episode of The Infra Pod, hosts Tim Chen (Essence VC) and Ian Livingstone (Keycard) sit down with Benny Chen, co-founder of Fireworks AI, to explore the evolving world of AI inference infrastructure.Benny shares his journey from Meta — where capacity planning meetings made it clear GPUs were heading "up and to the right" — to co-founding Fireworks AI before ChatGPT even launched. The conversation dives deep into why the team bet early on inference over training, how they approached model optimization from horizontal compiler techniques to per-model kernel tuning, and why model customization is the key to unlocking better-than-frontier performance for vertical use cases.Benny discusses the reality of open source vs. closed models, the rise of agentic workloads, and why the real question isn't which model to use — it's which tasks have already been saturated. This episode is packed with technical insights on inference infrastructure, reinforcement learning for model customization, and what it means to truly adopt an AI-native engineering culture.