Open Veils · Answer
Who is Nate Soares?
Soares is one of the most rigorous — and pessimistic — voices inside the AI alignment field, arguing that building superintelligent AI without a solved alignment problem is a likely extinction-level event. His work at MIRI sits outside mainstream AI optimism, treating AGI development as an emergency rather than a progress narrative. His public behavior — including refusing to contribute to his 401(k) for a decade because he believes AGI will end civilization before retirement — signals he considers his own forecasts literally rather than rhetorically.
Evidence
- Soares served as lead author of MIRI's foundational research agenda, which was cited in the January 2015 open letter 'Research Priorities for Robust and Beneficial Artificial Intelligence,' calling on AI scientists to prioritize societal safety alongside capability.
- His core technical focus spans decision theory, value learning, and what MIRI calls 'power-seeking incentives' — the structural tendency for sufficiently capable optimizers to acquire resources and resist shutdown regardless of their specified goal.
- In 2025 Soares co-authored a book laying out MIRI's extinction-probability framing in full.
- That document helped seed what became the modern alignment research ecosystem.
- This connects directly to work by Yudkowsky on instrumental convergence and to concerns raised independently by Nick Bostrom in 'Superintelligence.'