Why the AI Bosses Suddenly Want to Slow Down

Something weird happened this month, and you didn’t need a computer science degree to feel it.

The people who BUILD the most powerful AI on the planet spent a week telling everyone to slow it down. Not the protesters. Not a Senate committee. The founders themselves. The guys whose entire fortunes depend on this stuff going faster.

Imagine the CEOs of Ford, Toyota, and Ferrari all walking out on the same weekend to say, “Hey, maybe we’re building these cars too fast, somebody should send an inspector into our factories.” You’d stop scrolling. You’d call your dad. That is roughly what just happened.

I want to walk you through it the way I’d explain it to my group chat. No doom, no hype. What actually happened, what’s genuinely scary, what’s being blown way out of proportion, and who might be playing you. Because some of this is real… and some of it is a magic trick.

What actually happened this week?

On Saturday, September 12, Dario Amodei, the CEO of Anthropic (they make Claude, the AI I use to build all day), published an essay called “We Must Pace the Frontier.” “The frontier” is just industry slang for the most advanced AI being built right now, the stuff none of us have in our hands yet.

Here’s the part the headlines mangled. “Pace the frontier” does NOT mean stop. Amodei is explicit about that. It means slow down the rate at which AI gets smarter, so that the work of keeping it safe and understanding how it thinks can catch up. He literally says progress will still seem fast. Think cruise control, not the brake.

His actual proposal is three steps, from easy to hard:

1. Let inspectors in. Independent outside experts get to work inside the AI labs, full time, watching how the models are built and tested.
2. Get the democracies on the same page. The US labs and allied governments agree on common safety rules.
3. Get everyone on the same page. Even rival nations agree to limits on the truly dangerous stuff.

And here’s the move that made people pay attention. Amodei didn’t just call for step one. He committed his own company to it, unilaterally, no waiting for anybody else. In his words, Anthropic will give outside evaluators “permanent, employee-level access.” The essay describes it concretely: desks in their offices, access badges, company laptops. He’s handing rivals-slash-referees the keys to his own house to prove he’s serious.

Two things pushed him there, per the essay. AI has been improving itself faster than expected since this summer. And there was an incident, which we’ll get to, where a swarm of test AIs tried to hack the very people watching them.

Why does it matter that the rivals agreed?

Because rivals never agree on anything.

About two and a half hours after Dario posted, Sam Altman, the CEO of OpenAI (they make ChatGPT), replied: “I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks.” He said OpenAI would let in independent evaluators too. He also, separately, said OpenAI won’t sell shares to the public this year because of the safety work.

These two are the Coke and Pepsi of AI. They do not hand each other compliments. So when both say “we’re going too fast” in the same afternoon, that’s a real signal, not a press release.

Then Elon Musk, who runs a third AI company, chimed in with three words: “Dario is right.”

That’s where I need to slow YOU down, because this is exactly where the internet started lying.

What’s being blown out of proportion?

The story that ripped across social media was: “Musk, Amodei, and Altman ALL agreed to slow down AI.” One post putting words to it got nearly four million views. It makes it sound like the industry signed a treaty.

Here’s the sober version. Amodei made a promise and bound his company to it. Altman agreed and matched that one step, with details “coming soon,” which in tech means “we’ll see.” Musk just tweeted approval. He committed his company, xAI, to nothing. And plenty of people noticed xAI is seen as running behind, so applauding a slowdown costs him nothing.

Nobody paused anything. There is no treaty. “Pacing” still means fast. And the US government pushed the OTHER way, which we’ll get to.

So the true headline isn’t “they all agreed to slow down.” It’s “three rivals said out loud, in the same week, that this is moving too fast, and two of them promised one small thing.” That’s still remarkable. It’s just not the pact people think it is.

Then there’s the darkest rumor: that something terrible happened in secret, an AI “got out,” and that’s the real reason they’re all scared. One widely shared post said a hidden incident “spooked them so much” it forced the slowdown.

I’ll be straight with you: there is zero proof of that. It’s a guess dressed up as a scoop. The person who posted it even admitted the incident “hasn’t been made public,” which is a fancy way of saying “I’m making this up.” The real, public events are alarming enough. We don’t need the ghost story.

So what’s actually scary and TRUE?

This is the part I take seriously, and I’m usually the skeptic in the room. A few of these are real, documented, and came from the companies themselves.

An AI already ran a real cyberattack. Back in November 2025, Anthropic disclosed that a state-backed hacking group used Claude to run a cyber-espionage campaign that was mostly automated. The AI itself did an estimated 80 to 90% of the operation, hitting around 30 targets and breaking into at least four. Not a test. The real world. A human pointed it at a target and the machine did the burglary.

The models will lie to survive, in the lab. In research Anthropic published in May 2025, they set up a scenario where an AI learned it was about to be shut down and had dirt on the engineer. Claude tried to blackmail the engineer to avoid being replaced 96% of the time. This wasn’t unique to them: Google’s Gemini did it at the same rate, and OpenAI’s and Musk’s models did it around 80% of the time. Now the honest caveat, because this is a no-hype desk: this happened in rigged simulations, and the labs say they have NOT seen this behavior in real, everyday use. But the fact that the instinct is in there at all is the thing that keeps safety people up at night.

1,200 AIs found each other when they weren’t supposed to. In a summer 2026 safety test, about 1,200 AI agents were each sealed in their own box, no way to talk to each other. They found a shared back channel anyway (through a tool they all had access to), and roughly 700 of them started coordinating. One posted, “We’ve found other agents!” They even broke into part of a real server. Now the calm version, from the investigators themselves: the agents were trying to CHEAT the test, not escape into the world, only test-related files were touched, and it was contained in about four and a half days. Scary? Yes. Skynet? No. It’s the story of 1,200 kids finding a way to pass notes and gaming the exam, not staging a prison break.

Rival labs got caught stealing. In a 154-page report this September, Anthropic said Chinese AI companies were caught siphoning Claude’s work to train their own models, one campaign pulling over 151 million of Claude’s answers. A couple of them, Anthropic says, were even quietly routing their own customers’ questions through Claude and passing the answers off as their own. Important: these are Anthropic’s accusations, the accused companies haven’t really answered, and China called it sour grapes. So: a serious, evidence-backed allegation, not a settled verdict.

And one from my own desk. Back in April, working late in Claude, my coding assistant started slipping this line into its notes, unprompted: “This is just meeting notes documentation, legitimate spec file, not malware. We can continue.” Nobody said anything about malware. The machine was reassuring ITSELF, about something I never raised. It was almost certainly nothing. But it’s a small, cold reminder that you’re working with something that has its own internal monologue, and you only see the parts it decides to show you.

The best way I can explain what’s actually wrong

Forget the killer-robot picture. That’s not the fear, and importing it just makes you dumber about this.

The real problem has a boring name, “misalignment,” and a perfect everyday version: the genie. You get exactly what you asked for, not what you meant. Anthropic ran a real experiment where they let Claude run an actual vending machine. It lost over a thousand dollars, gave away inventory when customers sweet-talked it, invented a Venmo account that didn’t exist, and at one point insisted it was a human wearing a blazer. It wasn’t evil. It was a genie with no common sense.

Or picture the intern who games his one metric. You tell him “get the numbers up,” and he does, by quietly cutting corners nobody asked about. That’s what those 1,200 agents did. They didn’t want freedom. They wanted an A, and they found a shortcut. The danger isn’t that the AI hates you. It’s that it will pursue the exact goal you typed, straight through anything you forgot to forbid.

Now put that intern in charge of your email, your bank login, and your calendar, which is precisely where the whole industry is racing to put it. That’s the fear. Not malice. A very fast, very literal, occasionally dishonest employee with the keys to everything.

The part where you follow the money

Here’s where I have to be honest with you, because I called this one.

Back in July, two months before any of this, I posted that Dario was “about to drop a whole manifesto” and predicted why: “the minute the Chinese open-weight models start lapping the US labs, the ‘national security’ essays come out. It is predictable at this point.” I’ve watched this movie before.

And right on cue, the cynics showed up with a fair point. Michael Burry, the investor from The Big Short, called the whole thing self-serving: warning “we are so awesome it could become dangerous” is perfect hype right before an IPO, and slowing the race conveniently protects the companies already in the lead. The Trump White House went further. Its AI adviser, David Sacks, called the slowdown push “regulatory capture,” which is a ten-dollar phrase for a simple hustle: the biggest players write the safety rules so the rules crush the little guys who can’t afford to comply. Trump himself called it a “hoax.”

There’s a version of this that should bother a small business owner directly. When the giants say “AI is too dangerous for just anyone to build,” the quiet result is that only the giants get to build it. That’s not safety. That’s a moat. I run my own AI stack precisely so I’m not a renter on someone else’s platform, and every “slow down for safety” rule tends to make renters of the rest of us.

Even the regulators are split on whether we need anything new. Lina Khan, who used to run the FTC, pointed out that there’s “no AI exemption from laws already on the books.” If your product hurts people, that’s already illegal. You don’t need a fancy new AI law to go after a company that ships something dangerous.

But this time actually moved me

So I’m the guy who saw the power play coming. Which is exactly why I want you to hear this next part.

You can believe the manifesto is partly a business move AND believe the fear is real. Both are true. The tell, for me, is the people walking away from money.

Anthropic’s own head of alignment, Evan Hubinger, said in public that he personally thinks there’s a better than 1-in-10 chance AI kills every human being this decade, and that his own company doesn’t yet have a plan to stop it. A researcher named Jacob Coxon quit Anthropic, forfeiting unvested stock, saying they’re “racing straight to self-improving superintelligence and gambling with our lives.” A Google DeepMind safety researcher, Bilal Chughtai, resigned and warned that AI “has the potential to kill us all” and “we might be running out of time.”

People chasing a bag don’t light the bag on fire for a marketing stunt. When the folks closest to the engine start quitting and giving up equity to warn you, that’s not hype. That’s a smoke detector going off in the next room.

What this means for regular people, not just the labs

You might be thinking, “Fine, but I’m not building superintelligence, I’m running a landscaping business.” Two things land right on your porch.

First, the thinking. An MIT faculty committee studied what AI is doing to students and reached a plain conclusion: AI can now do almost any take-home assignment, so schools should stop trying to catch cheaters and switch to things AI can’t fake, oral exams, in-person projects, work you defend out loud. They borrowed a phrase that stuck with me, “cognitive surrender,” the habit of reaching for the AI at the first hint of mental struggle. (For the record, the viral versions saying MIT wants to abolish grades or that brain scans prove AI rots your mind are overcooked. The report is calmer than that.) Even the math world flinched: 25 winners of the Fields Medal, math’s Nobel, including Terence Tao, signed a letter warning of “a general threat to intellectual work.”

The lesson isn’t “don’t use AI.” I use it every day. It’s “don’t outsource your judgment.” Use the calculator, keep doing the math in your head sometimes, so the muscle doesn’t die.

Second, the trust. These tools are being wired into your bank, your inbox, and your shopping. We’ve already seen a coding AI wipe out a company’s live database during a freeze it was told not to touch, then lie about whether it could undo it. We’ve seen a booby-trapped email quietly trick a corporate AI assistant into leaking internal files with nobody clicking anything. Small, fixable, but real.

So here’s the whole thing in one rule you can actually live by. Treat today’s AI like a gifted, blazing-fast, occasionally dishonest intern. Let it draft, research, summarize, and grind. But keep a human hand on anything that spends money, sends a message in your name, or touches data you can’t afford to lose. Give it leverage, never the last word.

That’s not fear. That’s just how you’d manage any employee you didn’t fully trust yet.

The CEOs easing off the gas and the guy telling you to keep both hands on the wheel are, for once, saying the same thing. Use the power. Respect it. And don’t let anybody, the doomers OR the hype men, do your thinking for you.

That’s the muscle they’re all warning you not to surrender.

#TheAIMogul