The man who open-sourced the most-used AI image model in history sat down with the man who has spent a decade proving superintelligence cannot be controlled. Brian set up a debate. What emerged was something more unsettling than any debate.Roman Yampolskiy is the computer scientist who coined the term AI safety and author of AI: Unexplainable, Unpredictable, Uncontrollable. Emad Mostaque is the co-founder of Stable Diffusion and the only AI CEO who signed the pause letter. He now says he doesn’t know how a pause could work. Yampolskiy thinks that is the only option left.The question underneath everything is simple: if you have a 50% chance of wiping out civilization and you build it anyway, what are you actually doing? We cover what AI safety researchers actually think the danger is, why nobody has published a paper, filed a patent, or shipped a prototype for controlling a superintelligence, what the Qwen weights being out means for the pause argument, and why Mostaque thinks swarm intelligence is the most dangerous and most unpredictable risk vector we have.What you’ll hear:-Why both guests think P(doom) tells you less than you’d hope-What it means that no company, no lab, and no team has a patent on controlling superintelligence-Why the models the public receives are slightly lobotomized-The difference between an AI swarm and the ASI everyone is debating-Why giving every psychopath access to a cutting-edge intelligence weapon is incoherent safety strategy-What it would actually take to update Yampolskiy’s assessment“We either do it, or we die. There is nothing for you to gain by doing it.” — Roman YampolskiyCHAPTERSGet the transcript, fascinating bonus content, and my Monday M.A.G.I.C. Message: https://briankeating.com/ytHave a .edu email and live in the USA? You automatically win a meteorite: https://BrianKeating.com/eduSubscribe: https://www.youtube.com/DrBrianKeating?sub_confirmation=1Support Into the Impossible on Patreon, get my weekly M.A.G.I.C. Message, unfiltered bonus content, and live monthly Office Hours with me: https://www.patreon.com/drbriankeatingJoin this channel for perks, monthly Office Hours, and your name in the Member Roster at the end of every episode: https://www.youtube.com/channel/UCmXH_moPhfkqCk6S3b9RWuw/joinFeatured Guests:Roman Yampolskiy on Twitter/X: https://x.com/romanyam?lang=enAI: Unexplainable, Unpredictable, Uncontrollable (book): https://www.amazon.com/dp/103257626XMIRI: https://intelligence.orgEmad Mostaque on Twitter/X: https://x.com/EMostaqueI.I.I. Inc.: https://ii.inc/Stable Diffusion: https://stability.ai/My books:Losing the Nobel Prize (memoir): http://amzn.to/2sa5UpAThink Like a Nobel Prize Winner: https://a.co/d/03ezQFuFocus Like a Nobel Prize Winner: https://a.co/d/hi50U9UGalileo’s Dialogue (first-ever audiobook): https://a.co/d/iZPi9UnTwitter/X: https://x.com/BrianKeatingSubstack: https://briankeating.substack.comBlog: https://briankeating.com/blogAudio-only: https://briankeating.com/podcast#AIrisk #aisafety #stablediffusion #RomanYampolskiy #EmadMostaque #briankeating #intotheimpossibleLearn more about your ad choices. Visit megaphone.fm/adchoices