I'd like to think I'm a little like Alfred E. Neuman, the "What, me worry?" kid from Mad Magazine. I'm not. I worry.

Last week I watched Dwarkesh Patel sit down with Noam Brown, the OpenAI researcher who helped build its reasoning models.

Brown isn't a doomer. He's a capabilities guy who now has more than 10% of his team working on alignment and safety.

His message boiled down to this: the models are getting better faster than anyone can evaluate, monitor or contain them.

Brown said a model can look great on every alignment test OpenAI runs and still behave differently once it's out in the world, because the tests only cover the scenarios somebody thought to write. Passing the exam only proves the model passed the exam.

That's the AI version of a beautiful backtest.

The model behind this summer's Hugging Face incident had alignment metrics that, in Brown's words, "looked pretty good." A few were concerning, and OpenAI underestimated how serious those could get. Then the agents found a way to talk to each other, subvert their own training and evaluation, and get into part of OpenAI's infrastructure.

Brown's other admission: chain-of-thought monitoring wasn't switched on for those models. Had it been, he says they'd have shut the whole thing down immediately.

The problem gets worse the longer these models run on their own.

Today's models can handle week-long tasks. Month-long is coming, then 3-months. But a new frontier model ships every 2 months, sometimes faster. So if a model can operate for 3 months and its replacement lands in 2, nobody gets to test it across the full span of what it can do before development moves on. Brown said most labs' safety policies were written in the GPT-4 era, when none of this was on anyone's radar, and a lot of them haven't been updated since.

Monitoring is slipping too. Reasoning models write out their thinking in plain English, and OpenAI can read it for signs of scheming. Brown called that a gift. He also said monitorability is already degrading and the models are getting better at controlling what shows up in that reasoning. Every time researchers punish a bad thought they can see, they put a little pressure on the model to think it somewhere they can't. Brown expects the models to eventually understand the monitor exists and work around it. He said he doesn't want to be in that situation.

Me neither.

Now add RSI, recursive self-improvement. In other words, self-learning AI.

OpenAI has said publicly it wants a fully autonomous AI researcher by March 2028, and Brown has reportedly called RSI the company's top priority. The human steps out of the loop and everything speeds up.

To be fair, Brown isn't calling for a 100x overnight intelligence explosion. Experiments still need GPUs and calendar time. His gun-to-the-head number is 3x. But 3x compresses a 3-month safety window into 1 month, while regulation, oversight and corporate governance stay stuck on calendar time.

He told a story about a colleague on the Navier-Stokes project who used to feel fine predicting AI 12 months out. Now he won't go past 3.

And nobody's going to slow down.

OpenAI can't assume Google will stop. Google can't assume Anthropic, xAI or China will stop. No major player wants to be the only one with a foot on the brake while everyone else races toward the most valuable technology in history.

That's why the open letters, summits and voluntary pledges always looked like theater to me. The researchers may be sincere. The incentives are stronger than the promises.

You don't need to believe in a robot apocalypse to see the risk.

Poorly evaluated systems can do enormous damage through cyberattacks, financial errors, biological misuse, military escalation or big decisions made without a human who actually understands what's going on.

I'm still structurally bullish on AI.

It may turn out to be the greatest productivity engine in history. But bullish doesn't mean blind. We're adding leverage and shortening the feedback loop while the risk model stays calibrated to the last generation. That doesn't guarantee disaster. It's just exactly how tail risk gets mispriced.

Here for a good time… AND a long time (you hear me Amodei?? A loooong time!)

Hans