Show notes
Join us for an in-depth conversation with Rahul Rai, an Infrastructure Product Manager at Meta with over a decade of experience spanning Cisco, Samsung, Amazon, Microsoft, and Meta. In this episode, Rahul unpacks the complexities of AI model development—from training to inference—and reveals how product managers can navigate the technical and strategic challenges of building scalable systems that serve billions of users.What You'll Learn: • The fundamentals of AI training vs. inference and why distributed inference is a game changer.• How to balance the needs of multiple stakeholders—from ML engineers to end users—in building robust, scalable products.• Real-world insights into capacity planning and the creation of internal tools that impact global-scale operations.• A candid comparison of PM cultures and career paths at Amazon, Microsoft, and Meta, along with tips for transitioning into an infrastructure PM role.• Recommended resources and actionable advice for any PM looking to excel in high-impact technical roles.Timestamps:00:00 – Podcast Intro & Setup01:46 – Meet Rahul Rai: Background and Career Journey03:17 – AI 101: Training vs. Inference Fundamentals06:24 – Deep Dive: Analogies and Real-World Examples of AI Inference10:03 – Distributed Inference Explained: How Models Stay Current12:34 – Cost Breakdown: Why Inference Drives 90% of AI Model Costs17:13 – The Infra PM Role: Balancing Stakeholder Needs22:43 – Building at Scale: Capacity Planning Tools for Billion-User Platforms24:36 – Meta’s AI Strategy: Open Source Models and Product Integration28:28 – Comparing Cultures: PM Roles at Amazon, Microsoft, & Meta36:50 – Autonomy at Meta: Bottom-Up Problem Solving in Action40:30 – Growth Opportunities: Essential Skills for Infra PMs42:46 – Measuring Success: Metrics and Impact in Infrastructure PM44:32 – Resources for PMs: "First 90 Days" & "Getting Stuff Done"46:05 – Wrap-Up & Connect: How to Follow Rahul RaiRecommended Tags:#ProductManagement #InfrastructurePM #AIInference #DistributedInference #Meta #Amazon #Microsoft #TechLeadership #CapacityPlanning #OpenSourceAI #MachineLearning #ProductManager #TechPodcast
