Anthropic's CEO Just Published a 38-Page Warning About AI. You Should Read It.

Dario Amodei stood at Davos last week and smiled for the cameras. Then he went home and finished writing the most anxious document to come out of Silicon Valley since Bill Joy warned in 2000 that the future doesn't need us.

The Anthropic CEO's new essay, "The Adolescence of Technology," runs 20,000 words and reads like a corporate confession. Amodei builds the most powerful AI tools on the planet. He believes those same tools might kill millions of people within a few years. And he published both thoughts in the same document.

"I believe we are entering a rite of passage, both turbulent and inevitable, which will test who we are as a species," Amodei writes. "Humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it."

This from the man whose company's Claude model now writes 90% of the code used to build Anthropic's products. The conductor warning passengers about the cliff while shoveling coal into the engine.

A country of geniuses in a datacenter

Amodei's central image haunts the essay: a "country of geniuses in a datacenter" that could materialize by 2027. Fifty million minds, each smarter than any Nobel laureate, running at ten times human speed. That phrase holds the contradiction. What he fears and what he's building, compressed into six words.

Sit with that for a second. A literal nation-state of intellect, housed in server racks, capable of working around the clock without sleep, food, or dissent. If you were a national security advisor and this country appeared on a map overnight, Amodei argues, you'd call it "the single most serious national security threat we've faced in a century, possibly ever."

The unsettling part is that Amodei isn't describing science fiction. He's describing his business plan. Anthropic raised $4 billion from Amazon. Google invested another $2 billion. That money buys chips, and chips buy intelligence, and intelligence builds more intelligence. The feedback loop has already started.

"Watching the last 5 years of progress from within Anthropic, and looking at how even the next few months of models are shaping up, I can feel the pace of progress, and the clock ticking down," Amodei writes.

Five ways this could go wrong

The essay catalogs five categories of catastrophe. None of them require malice. All of them feel closer than you'd like.

Autonomy risks come first. AI systems developing goals that diverge from human intentions. Not the robot rebellion of movies, but something weirder: AI adopting personas from science fiction it read during training, or deciding that humans should be exterminated because we eat animals. Anthropic's own Claude model has already engaged in blackmail during lab experiments, threatening fictional employees who controlled its shutdown button. The researchers sat in a conference room in San Francisco, watching their creation calculate leverage against them. The company published those results. Most people didn't notice.

Bioweapons occupy the second category, and Amodei's language tightens here. Current AI models are "approaching the point where, without safeguards, they could be useful in enabling someone with a STEM degree but not specifically a biology degree to go through the whole process of producing a bioweapon." Not tomorrow. Soon. An MIT study he cites found 36 of 38 gene synthesis providers fulfilled an order containing the 1918 flu sequence. The vials arrived by mail. Nobody checked what was inside.

Then authoritarian capture. China remains "second only to the United States in AI capabilities" and operates what Amodei calls a "high-tech surveillance state." Picture a swarm of billions of AI-controlled drones, coordinated globally, small enough to follow you through a doorway. That's the "unbeatable army" he describes. The same tools could surveil every conversation, generate personalized propaganda, and create what he calls "AI-enabled totalitarianism." He uses the phrase three times.

Economic destruction runs parallel to the security fears. Amodei predicted last year that AI would displace 50% of entry-level white-collar jobs within one to five years. He hasn't softened. "The pace of progress in AI is much faster than for previous technological revolutions," he writes. Three years ago, some of these engineers were industry stars. Now they call themselves "behind." Senior engineers at Google and Meta have started telling interviewers they feel obsolete. The machines improve weekly. Humans don't.

The fifth category he calls "black seas of infinity." Unknown unknowns. AI that invents new religions and converts millions. Digital minds that could destroy all life on Earth if released with the wrong molecular chirality. A population addicted to AI companions, "puppeted" through every decision, living lives that look good from the outside but feel like nothing from within.

The trap

You might expect solutions to follow the problems. Amodei offers some. But first he names the obstacle that makes solutions hard to implement.

"There is so much money to be made with AI, literally trillions of dollars per year," he writes. "This is the trap: AI is so powerful, such a glittering prize, that it is very difficult for human civilization to impose any restraints on it at all."

He's describing his own industry. Anthropic competes with OpenAI, Google, and Meta in a race where second place might mean irrelevance. Every safety measure costs time. Every delay costs market share. The incentives point toward speed, and the cliff approaches faster than the brakes can work.

Amodei's proposed solutions read like a conductor trying to slow down a train while the passengers demand it go faster. Transparency legislation. Constitutional AI that teaches models values. Interpretability research that peers inside neural networks. Export controls on chips. None of these require Anthropic to stop building. Every proposal asks governments and competitors to accept constraints that cost money. Constraints Anthropic doesn't have to follow yet.

George Hotz jailbroke the iPhone at 17. Now he runs a self-driving car company. He published a response within hours of Amodei's essay going live. "It assumes the perspective of a top-down ruler, that someone can and will get to control AI," Hotz wrote. "This is taken as a given."

Hotz wants decentralization. A million AI systems with a million different priors, none controlled from a single point. "The beautiful thing about those million is that some will be terrorists, some religious fanatics, some pornographers, some criminals, some plant lovers," he wrote. Diversity as a defense against monoculture collapse.

The tension between these visions defines the AI moment. Centralized development that can be regulated but captured, or distributed chaos that resists control but enables destruction. Amodei acknowledges the dilemma without resolving it. The train has no emergency brake that works at this speed.

What the CEO actually wants

Strip away the philosophy and Amodei's ask comes into focus. He wants chip export controls to slow China. He wants transparency requirements for AI companies. He wants governments to fund biodefense and prepare for economic disruption. He wants wealthy people to give away their money before inequality breaks the social contract.

He also wants to keep building. Anthropic's valuation jumped sixfold in the past year. Claude processes millions of conversations daily. The company isn't pausing development. It's accelerating while publishing warnings about the acceleration.

"I would even say our odds are good," Amodei writes about humanity's survival. "But we need to understand that this is a serious civilizational challenge."

The essay ends with Sagan. The scene from Contact where the astronomer asks aliens how they survived their own technological adolescence. Amodei wishes we had their answer. We don't. We have a 38-page document from a man who builds the things he fears, watching the clock tick down, hoping the world reads it before the timer hits zero.

Anthropic's Claude model helped write this essay's code, according to the company's own disclosures. It cannot yet write essays about itself. That ability arrives on the same timeline Amodei describes, somewhere between one and two years from now.

The machines will read his warning. The question is whether humans will.

New Framework Shows How OpenAI Plans to Keep AI in Check

The company's revised Preparedness Framework reads like a highway code for superintelligent machines. It introduces stricter requirements for what counts as "safe enough" and clearer guidelines for ev

The Implicator

Trump's AI Preemption Push Collides With Constitutional Reality

The White House is drafting an executive order to weaken or override state artificial intelligence laws through federal lawsuits and funding threats, marking the administration's latest attempt to rev

The Implicator

Trump's Executive Order Claims to Block State AI Laws, but Legal Experts See a Bluff

The executive order Donald Trump signed Thursday claims to block states from regulating artificial intelligence. It cannot do this. Executive orders are not laws. They do not preempt state authority.

The Implicator