Episode 124You may think you’re doing a priori reasoning, but actually you’re just over-generalizing from your current experience of technology.I spoke with Professor Seth Lazar about:* Why managing near-term and long-term risks isn’t always zero-sum* How to think through axioms and systems in political philosphy* Coordination problems, economic incentives, and other difficulties in developing publicly beneficial AISeth is Professor of Philosophy at the Australian National University, an Australian Research Council (ARC) Future Fellow, and a Distinguished Research Fellow of the University of Oxford Institute for Ethics in AI. He has worked on the ethics of war, self-defense, and risk, and now leads the Machine Intelligence and Normative Theory (MINT) Lab, where he directs research projects on the moral and political philosophy of AI.Reach me at [email protected] for feedback, ideas, guest suggestions. Subscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSSFollow The Gradient on TwitterOutline:*

Intro*

Ad read — MLOps conference*

The allocation of attention — attention, moral skill, and algorithmic recommendation*

Attention allocation as an independent good (or bad)*

Axioms in political philosophy*

Explaining judgments, multiplying entities, parsimony, intuitive disgust*

AI safety / catastrophic risk concerns*

Superintelligence arguments, reasoning about technology*

Attacking current and future harms from AI systems — does one draw resources from the other? *

GPT-2, model weights, related debates*

Power and economics—coordination problems, company incentives*

Morality tales, relationship between safety and capabilities*

Feasibility horizons, prediction uncertainty, and doing moral philosophy*

What is a feasibility horizon? *

Safety guarantees, speed of improvements, the “Pause AI” letter*

Sociotechnical lenses, narrowly technical solutions*

Experiments for responsibly integrating AI systems into society*

Helpful/honest/harmless and antagonistic AI systems*

Managing incentives conducive to developing technology in the public interest*

Interdisciplinary academic work, disciplinary purity, power in academia*

How we can help legitimize and support interdisciplinary work*