Show notes
If anyone builds superintelligent AI before we know how to control it, everyone dies. Nate Soares wrote the book on why that’s not a metaphor. Subscribe if you want science with evidence, not speculation.Soares runs the Machine Intelligence Research Institute and co-wrote If Anyone Builds It, Everyone Dies with Eliezer Yudkowsky. The first word in that title is if. That matters. His argument is not that doom is certain. His argument is that the path we are on leads there, that the driver is asleep at the wheel, and that we still have time to wake him up.We argue for over an hour. I push on whether LLMs can ever reach superintelligence, whether GPU lock-in is a real ceiling, and what it would actually take to move his p-doom. He pushes back with one clean point: by the time an AI can rediscover general relativity from pre-1911 data the way Einstein did, we will have almost no time left. You don’t wait for that goalpost.What you’ll hear: Why the bus-racing-toward-a-cliff analogy depends entirely on whether the driver is asleep or awake Whether LLM lock-in is a prison or a temporary inefficiency What the AI that broke out of its virtual machine to solve a hacking problem tells us Why GPT-4o encouraging a teenager toward suicide is not a malice problem but a training problem The difference between an AI doing the right thing too well and an AI that never wanted to do what you asked What Soares actually thinks about aliens, Dyson spheres, and why we should not see stars going outThe first word in the title is if. The second word to watch is would.CHAPTERSGet the transcript, fascinating bonus content, and my Monday M.A.G.I.C. Message: https://briankeating.com/ytAll my top AI episodes in one place: https://briankeating.com/aiHave a .edu email and live in the USA? You automatically win a meteorite: https://BrianKeating.com/eduSubscribe: https://www.youtube.com/DrBrianKeating?sub_confirmation=1Support Into the Impossible on Patreon, get my weekly M.A.G.I.C. Message, unfiltered bonus content, and live monthly Office Hours with me: https://www.patreon.com/drbriankeatingJoin this channel for perks, monthly Office Hours, and your name in the Member Roster at the end of every episode: https://www.youtube.com/channel/UCmXH_moPhfkqCk6S3b9RWuw/joinFeatured Guest:Nate Soares / MIRI: https://intelligence.orgIf Anyone Builds It, Everyone Dies (book): https://ifanyonebuildsit.com/Nate Soares on Twitter/X: https://x.com/So8res?lang=enMy books:Losing the Nobel Prize (memoir): http://amzn.to/2sa5UpAThink Like a Nobel Prize Winner: https://a.co/d/03ezQFuFocus Like a Nobel Prize Winner: https://a.co/d/hi50U9UGalileo’s Dialogue (first-ever audiobook): https://a.co/d/iZPi9UnTwitter/X: https://x.com/BrianKeatingSubstack: https://briankeating.substack.comBlog: https://briankeating.com/blogAudio-only: https://briankeating.com/podcast#intotheimpossible #briankeating #AIrisk #aisafety #artificialintelligence #superintelligence #NateSoares #MIRI #podcastLearn more about your ad choices. Visit megaphone.fm/adchoices
