As head of Product Management for Generative AI at Meta, Joe Spisak leads the team behind Llama, which just released the new 3.1 405B model. We spoke with Joe just two days after the model’s release to ask what’s new, what it enables, and how Meta sees the role of open source in the AI ecosystem.Joe shares that where Llama 3.1 405B really focused is on pushing scale (it was trained on 15 trillion tokens using 16,000 GPUs) and he’s excited about the zero-shot tool use it will enable, as well as its role in distillation and generating synthetic data to teach smaller models. He tells us why he thinks even frontier models will ultimately commoditize—and why that’s a good thing for the startup ecosystem.Hosted by: Stephanie Zhan and Sonya Huang, Sequoia Capital Mentioned in this episode: Llama 3.1 405B paperOpen Source AI Is the Way Forward: Mark Zuckerberg essay released with Llama 3.1.Mistral Large 2The Bitter Lesson by Rich Sutton