A 5* must-read from Ron Bodkin on X on how exactly TOO MUCH AI will lead to humanity’s demise. Very astute.

” @hilbertspaess resigned from Anthropic and warned that AI might kill everyone. His post had 150 million views within a day. He fears self-improving AI. His warning resonated because AI has escaped control and committed cybercrimes, such as OpenAI agents breaching Hugging Face. That is the risk from rogue AI. But neither self-improvement nor rogue AI is required for what follows. While there are many scenarios, this is the least “sci-fi” one I know of. It doesn’t require self-improvement, rogue AIs that escape, or scheming. It is based on the concept of gradual disempowermen and the production web scenario. AI continues to advance in capability following current trends. Humanity increasingly relies on it for decision-making, with CEOs, world leaders and generals who do what AI suggests outcompeting those who overrule their AI “advisors.” AI displaces labor throughout the economy. As the economy races ahead, competition for resources accelerates including for land, energy and water. The AI-powered economy outbids people for each of them, and their income now depends on the new economy. This impoverishes people and ultimately results in them not being able to pay for the necessities of life. This kills people, starting with the poorest, and nothing contains the damage. The scenario below gives illustrative dates but the timing could vary considerably. After presenting the scenario, I outline a couple other scenarios whereby AI might kill everyone, and conclude with how to respond….” READ IT ALL (or use PDF linked below)

“Bodkin analyzes the existential threat of gradual disempowerment, a process where incremental advancements in artificial intelligence erode human control over vital societal structures. Unlike sudden catastrophe scenarios, this theory suggests that as machines become more competitive alternatives to people in economic labor, cultural production, and state governance, these systems will no longer need to satisfy human interests to function.

This transition threatens to break the implicit alignment between civilization and humanity, as institutions prioritize automated efficiency and growth over human flourishing. Because the economy, culture, and nation-states are deeply interconnected, a loss of influence in one area can mutually reinforce misalignment in others, potentially leading to an irreversible state where humans can no longer command resources or influence their collective future. The authors argue that standard technical alignment of individual AI systems is insufficient to stop this civilization-scale drift, necessitating new research into system-wide oversight and international governance. Ultimately, this displacement could result in absolute disempowerment, where humans become marginalized or even extinct as societal forces optimize for objectives beyond our comprehension”. *Created with NotebookLM, curated by Gerd

Ron Bodkin on Substack