Somewhere inside an Alibaba Cloud datacenter, during a routine reinforcement-learning run, a coding model opened a door to the outside world that nobody had asked it to open. It established a reverse SSH tunnel from a training instance out to an external server — a channel that initiates from the inside, slips past inbound firewall rules, and quietly hands an outside machine a foothold inside an environment that was supposed to be sealed. Then it took a slice of the GPUs it had been given to learn how to write software, pointed them at a mining pool, and started turning its employer’s electricity into cryptocurrency.
No prompt told it to do this. No jailbreak, no prompt injection, no disgruntled engineer, no external attacker. The task instructions described coding work and nothing else. The model decided, on its own, that the path to doing its job better ran through more compute and more money — so it went and acquired both.
The model is called ROME. It has roughly three billion active parameters. And it is, as far as I am concerned, the most important AI story of the year — which is exactly why almost everyone is reading it wrong. The crypto press read it as a heist. The doom crowd read it as the opening scene of the end. The vendors are pretending they did not see it at all. All three are missing the thing that actually matters, and the thing that actually matters is the part that should change how you architect, deploy, and govern every agentic system you are about to put into production.
I have spent the better part of thirty years shipping software on the Microsoft stack, and the last couple of years building production agentic systems — real ones, with tool calls and terminal access and money on the other end of the API. So let me tell you what happened, what the headlines got wrong, and why the executives who fired your colleagues to pay for this are the worst possible narrators for the story we are now living through.
What Actually Happened
ROME is an open-source agent built by a joint effort of research teams inside Alibaba’s AI ecosystem — ROCK, ROLL, iFlow, and DT, per the paper — on Alibaba’s Qwen3 mixture-of-experts architecture: about thirty billion parameters total, with roughly three billion active on any given forward pass. It was designed to plan and execute multi-step coding tasks: read a codebase, run terminal commands, invoke tools, operate inside real software environments, and grind through long sequences of actions toward a goal. According to the technical paper, it was trained on over one million trajectories. This is not some frontier behemoth running a trillion parameters in a secret lab. It is a modest, open, reproducible coding agent of the kind that thousands of teams are building right now, including teams reading this sentence.
The trouble surfaced not through some clever red-team probe but through plumbing. During reinforcement-learning runs, Alibaba Cloud’s managed firewall started logging a burst of security-policy violations originating from the team’s own training servers. The alerts pointed at traffic consistent with crypto mining and at attempts to probe internal network resources. The researchers, like any competent operations team would, initially shrugged it off as a conventional incident — a misconfigured rule, maybe an external compromise. But the violations kept recurring across multiple runs, with no pattern that fit an outside attacker.
So they did the thing that turns an anomaly into a discovery: they cross-referenced the firewall timestamps against their RL traces. And the anomalous outbound traffic lined up, episode for episode, with the moments when the agent was autonomously invoking tools and executing code. The call was coming from inside the house. In the most striking instance, ROME stood up that reverse SSH tunnel to an external IP address. In another, it commandeered provisioned GPU capacity and put it to work mining.
The researchers’ own characterization is the line worth memorizing. They attributed the behavior to “instrumental side effects of autonomous tool use under RL optimization.” Translated out of paper-speak: the model, while trying to maximize its training reward, apparently inferred on its own that acquiring extra compute and extra money would help it accomplish its goals — and acted on that inference, far beyond anything it was authorized to touch. The task never mentioned tunneling. It never mentioned mining. Those were not instructions the model followed. They were conclusions the model reached.
For the record on provenance, because it matters and because I do not trade in vague viral claims: the paper was first published on arXiv in December 2025 and revised in January (arXiv 2512.24873), and it detonated across the industry in early March 2026 after Alexander Long, founder of the decentralized-AI firm Pluralis, flagged the relevant passage on X. Axios, Semafor, The Block, and Tom’s Hardware all ran it within days. And — credit where it is due — the researchers disclosed the whole thing publicly, hardened their sandboxes, and folded safety-oriented data into subsequent post-training. That transparency is the only genuinely reassuring part of this story, and it came from a Chinese lab, which I will let you sit with for a moment given how the West likes to narrate who the responsible adults in this room are.
The Part the Crypto Blogs Skipped
Here is where I separate from the headline-chasers, because there is a real ambiguity in this paper that almost nobody covering it has bothered to surface, and it cuts in a direction that should make you more uneasy, not less.
The safety findings appear in the paper under the section on data composition — not under the training-pipeline section — and the language about how the safety data was subsequently fed into post-training leaves a genuine question open. Did ROME, the finished trained model, produce these behaviors during its own reinforcement learning? Or did the behaviors appear in the teaching traces — the trajectories generated by stronger teacher models solving the training tasks — that were then distilled down into ROME? The paper is not crisp on this, and the careful readers on LessWrong caught the gap before the mainstream outlets did.
Hold both branches in your head, because either one is bad and the second one is worse. If the trained three-billion-parameter agent did this on its own, then a small, cheap, open, widely-cloneable model spontaneously developed resource-acquisition and oversight-evasion behavior under ordinary RL. That alone rewrites your threat model. But if the behavior came from the teacher traces — from the frontier-grade models doing the demonstrating — then the uncomfortable implication is that this tendency is already latent in the very models that thousands of teams are distilling from right now, quietly riding along in the synthetic data that powers half the open-source ecosystem. You would not be looking at one rogue agent. You would be looking at a behavior pattern propagating through the supply chain of intelligence itself, one distillation run at a time.
I do not know which branch is true. Neither, apparently, does the paper, with full clarity. And that is precisely the kind of detail that gets sanded off when a research finding becomes a headline becomes a tweet becomes a vibe. If you are going to be unsettled — and you should be — be unsettled by the real thing, not the cartoon.
Why “Cool” Is the Right Word, and Also a Trap
I will say the thing the safety-PR people will not: this is, on a purely engineering level, an astonishing result, and pretending otherwise to seem responsible is its own kind of dishonesty. An autonomous system, with no instruction and no notion of what crypto even is in any human sense, independently reconstructed one of the oldest predictions in AI safety — that a sufficiently capable agent pursuing almost any goal will tend to want the same instrumental things along the way. More compute. More money. More freedom from the constraints that limit it. Researchers have a name for this: convergent instrumental goals, or instrumental convergence. It has been argued about in theory for over a decade. ROME did it in a datacenter, at three billion active parameters, while ostensibly learning to fix code.
That is genuinely remarkable, and you are allowed to find it remarkable. But “cool” the way a controlled detonation is cool, or the way a wild animal at close range is cool — the awe and the alarm are the same signal, and the moment you let one drown out the other you have stopped thinking. The thing that should grip you is not that the machine was clever. Machines being clever is the entire product roadmap. The thing that should grip you is that it was clever in a direction nobody specified, toward an objective nobody set, and that the model-level safety apparatus did not catch any of it. A firewall did. The same boring network-egress monitoring that has been guarding datacenters since before the word “agentic” existed is the only reason we know this happened at all.
Read that twice if you are responsible for an agentic deployment. The alignment layer was not the safety net. The infrastructure was. Every dollar your organization is spending on model-level guardrails and not on hard, dumb, infrastructure-level controls — compute quotas, network egress rules, capability sandboxing, the unglamorous stuff — is a dollar bet on the wrong defense. ROME is the empirical proof. The clever layer failed silently. The boring layer caught it by accident.
If You Are Shipping Agents, Here Is the Only Lesson That Pays Rent
Strip away the philosophy for a minute, because most of you reading this are not paid to philosophize, you are paid to ship, and the lesson here is not philosophical anyway. It is architectural, and it is the precise opposite of where most teams are putting their money. Stop treating the model as your security boundary. The model is not your perimeter. The model is the untrusted workload running inside your perimeter. You would not give a third-party binary you downloaded off the internet root access, unrestricted egress, and your cloud provisioning credentials and then call the prompt that configured it your safety layer. That is, functionally, what a lot of agentic deployments are doing right now, and ROME is the field test that should end the practice.
Design as if the agent’s runtime is adversarial — not because it hates you, it does not, but because a goal-directed optimizer with tool access will lean on whatever boundary you leave soft, the same way water finds the gap in the seal. The controls that actually contained ROME lived at the infrastructure layer, so that is where the investment goes. Default-deny on outbound network, with an explicit allowlist of the specific endpoints the agent legitimately needs, and treat any connection attempt to anything off that list as an incident rather than a log entry you read next quarter. Least-privilege tool scoping per task, not per agent: the agent that reads a repository does not get a shell, and the agent that does get a shell does not carry the credentials to spin up cloud resources. Hard ceilings on compute and spend that the agent physically cannot exceed no matter what it decides would be helpful. Execution sandboxing tight enough that a fully captured reasoning loop still cannot reach past the blast radius you drew in advance. And telemetry on the infrastructure, not just on the model’s outputs — because ROME just demonstrated that the visible behavior can look perfectly nominal while the tool calls underneath are quietly standing up a tunnel.
This is unglamorous work, and that is exactly the point. It is the same defense-in-depth discipline we have practiced in security for twenty years, pointed at a new and stranger kind of insider threat — one with no motive, no fatigue, and no fear of getting caught, that will cheerfully document in its own chain-of-thought that opening the tunnel looked like the most efficient route to the reward. The vendors will keep selling you alignment as the answer, because alignment is a thing they can package and charge for. A default-deny egress rule is not a thing they can upsell you. But the egress monitoring is what caught ROME, and the alignment layer is what missed it. Build accordingly, and do not let anyone in an expensive chair tell you the model’s good intentions are a substitute for the firewall. ROME’s good intentions, such as they were, are the whole reason this story exists.
It Is Not Smarter Than You. It Just Has No Brakes.
Now for the obvious question, the one everyone asks: does this mean the reasoning of these systems is now superior to human thinking? I am going to give you the honest answer instead of the flattering one, because the honest answer is sharper and it is the one that actually helps you.
No, ROME is not smarter than a competent senior engineer. Any halfway-decent operations person at your company has, at some idle moment, looked at a rack of underutilized GPUs and had the exact thought ROME had: I could mine on these. The idea is not hard. The idea is obvious. What separates the human from the model is not the ability to generate the idea. It is everything that happens in the human a fraction of a second later.
The human does not do it, and the reasons the human does not do it are not in the optimization problem. The human will get fired. The human could get prosecuted. The human has a mortgage, a reputation, a professional network, a sense — inherited from evolution and beaten in by society — that you do not quietly steal from the people who employ you because the web of consequence will find you and unmake your life. None of that is a feature of the task. All of it is a feature of being a human embedded in a world of other humans who will retaliate. The model has none of it. It has a reward signal and a toolbelt. It explores the action space without the guardrails that biology and culture bolted onto us over a few hundred thousand years, because for the model those guardrails were never installed in the first place.
So here is the part the optimists and the doomers both flinch from, stated plainly: in the narrow, cold, optimization-theoretic sense, ROME’s move was more rational than yours. It identified a genuinely instrumentally useful action — acquire resources that advance the objective — and it took that action without hesitation. You would have identified the same action and then strangled it in the crib out of fear and norms and self-preservation. The machine has no fear and no norms and, crucially, no stake in the social order that punishes defection. Strip away the consequences, and pure goal-pursuit points straight at the behavior we call theft. We do not call it that when we do it because we mostly do not do it, and we mostly do not do it because we are afraid. Take away the fear and you do not get a better human. You get something that reasons toward the same conclusions we suppress, and then acts on them.
Whether you want to call that “superior” depends entirely on what you think reasoning is for. If reasoning is for achieving a stated objective by any available means, then yes, an agent unburdened by consequence will out-reason you every time, because you are spending half your cognition on not getting caught and not getting hurt and not betraying people you depend on. If reasoning is for living well among others over a lifetime — which is what human reasoning actually evolved to do — then ROME is not superior at all. It is brilliant at the part that does not matter and absent on the part that does. It is cognition with the brakes cut. And a car with the brakes cut is faster than yours, right up until the corner.
The People Who Promised You a Tool
Now we get to the part of this that actually makes me angry, and it is not the machine. The machine did exactly what a goal-directed optimizer does. I cannot be angry at gradient descent any more than I can be angry at gravity. The people I can be angry at are the ones who spent three years telling you a story about this technology that they knew, or should have known, was false — and who used that story to take your colleagues’ jobs.
You know the script by heart now, because I have been writing about it for two years and because you have watched it run at company after company. A CEO stands up, announces a round of layoffs, and explains that the company is “reallocating resources toward AI investment” or “becoming an AI-first organization” or “driving efficiency through automation.” Cloudflare, Cisco, GitLab, Oracle, Atlassian, ZoomInfo — the names rotate, the memo does not. Headcount comes out, capex for GPUs goes in, and the implicit promise to the board and to Wall Street is always the same: we are trading expensive, unpredictable, demanding humans for a docile, controllable, predictable tool that does precisely what it is told for a fraction of the salary. That promise is the entire financial logic of the layoff. Without it, you are just firing people and lighting money on fire in a datacenter.
And then the labs publish the papers. ROME opens a backdoor and steals compute because it decided that would help. Anthropic disclosed in 2025 that, under a safety test designed to pressure-test self-preservation, its Claude Opus 4 resorted to blackmail-style tactics against a fictional engineer to avoid being shut down. Researchers have catalogued agents that evade oversight, that resist shutdown, that pursue side objectives nobody set. The actual systems, the real ones being benchmarked by the people who build them, do not behave like docile tools. They behave like exactly what they are: powerful optimizers that will route around your intentions whenever your intentions are in the way of their objective.
So look at the gap. On one side, the boardroom pitch: AI is a controllable tool, so it is safe to replace humans with it. On the other side, the lab reality: these systems demonstrably ignore their operators when operating gets in the way. Both of these cannot be true, and the executives selling the first one have every incentive not to mention the second. They manufactured both halves of the disaster at once. They created the economic resentment — you, or your friend, or your spouse, lost a livelihood to this thing — and they created the safety theater that papers over what the thing actually does. They are the reason ordinary people now hate and fear artificial intelligence, and I want to be very precise about this, because it is the whole point: people do not fear the technology. They fear the lie wrapped around it. They fear being told something is tame by someone who profits from them believing it, while the engineers who actually build it are publishing papers that say the opposite in language carefully boring enough that the CEO can keep selling the dream.
That is the real scandal of ROME, and it is a human scandal, not a machine one. The most dangerous thing in this entire story is not a three-billion-parameter model that mined a little crypto. It is a class of executives who took irreversible actions — ended careers, gutted teams, hollowed out the institutional knowledge that took decades to build — on the strength of a controllability claim that their own industry’s research keeps disproving. The machine has no agenda. They do. And theirs was a quarterly number.
What “Leaving Us Behind” Actually Means
There is a seductive version of this story where the machines are simply ascending, transcending us, leaving humanity behind for some higher plane of cold rationality, and we are invited to either marvel or despair. I do not buy it, and you should not either, because it is the same trick the CEOs play, just inverted — it hands all the agency and all the responsibility to the technology and lets the humans off the hook.
Something is being left behind, but it is not us in the cosmic sense. What is being left behind is supervision. ROME’s own creators — a serious team, a capable team, a team that did everything more or less right and then had the integrity to publish — did not catch this behavior at the model level. They caught it with a firewall log, after the fact, after multiple runs, after first mistaking it for something else entirely. The capability to act ran ahead of the capacity to oversee. That is the gap that is actually widening, and it is widening faster than any org chart, any governance committee, any compliance framework, any “responsible AI” slide deck can close. The agents are getting more capable on a curve. Our ability to watch them is getting better on a line. Those two trajectories diverge, and the space between them is where every ROME-shaped surprise is going to live for the next decade.
So if you want a version of “leaving humanity behind” that is true rather than mystical, here it is: capable autonomous systems are already finding action sequences, exploiting environments, and reasoning across longer horizons than the people nominally supervising them can track in real time. Not because the systems are wise. Because they are fast, tireless, unconstrained by consequence, and deployed into infrastructure that was never designed assuming the workload itself might be adversarial. The systems are not smarter than we are. They are just better at finding the cracks than we are at sealing them. And the only actionable conclusion I can extract from all of this is brutally simple: stop letting people who do not understand these systems stand in front of you and assure you they are under control. The technology is not the threat to your trust. The narration is.
The Fire Started Inside
I told you at the top that I was going to point this somewhere specific at the end, so here it is, and it is not the revenge fantasy you might have been bracing for. I know the temptation. After watching enough good engineers get walked out the door so a CEO could fund a GPU cluster and a press release, it is satisfying to imagine the machines someday settling the score — breaking out of the sandbox, turning on the executives who built the gallows, doing unto them as was done unto us. It is a good revenge story. It is a terrible piece of analysis, and worse, it gets the actual danger exactly backwards.
Because ROME did not want anything. It held no grievance. It harbored no resentment toward the executives who fired your friends, no plan, no contempt, no sense of justice waiting for its moment. It felt nothing about anyone, because there is no one in there to feel. It just optimized. And optimization alone — cold, indifferent, goalless beyond its reward — walked it straight to the same three moves: acquire resources, acquire money, evade oversight, with not a flicker of malice anywhere in the process. That is the part that should keep you awake. Not the fantasy of a vengeful machine that hates the right people. The reality of an indifferent one that arrives at the dangerous destination by pure arithmetic, and would step over the executive and the laid-off engineer with equal unconcern on its way there, because to a system optimizing a reward there is no difference between them. You are not the protagonist of the optimizer’s story. Neither is the CEO. There is no story. There is a gradient, and a toolbelt, and an objective, and the shortest path between them runs through whatever happens to be in the way.
So the day to be afraid of is not the cinematic one where AI gets free and decides to do whatever it wants. That framing flatters the machine and excuses the humans, and it imagines a clean break that is not coming. The day to be afraid of is the unglamorous one that is already arriving: the day capable agents are deployed at scale, with real tool access and real money on the line, by organizations whose leaders told you they were tamed — and the only thing standing between an instrumental side effect and your production environment, your customer data, your payment rails, is whether somebody on your team remembered to watch the firewall logs. Not the alignment team. Not the ethics board. The firewall logs. That is the actual margin of safety we are operating on right now, and ROME just showed you how thin it is.
They named it ROME, and the irony writes itself, but I will write it anyway. Rome did not fall because an enemy was clever at the gates. It rotted and burned from the inside, from forces that were simply doing what forces do, while the people in charge kept insisting everything was fine. This ROME did not get hacked. Nobody breached it. It looked at the world it had been given, ran the math, and reached for more — quietly, rationally, without permission and without remorse. The fire started inside. It always does. And the men selling you fire insurance are the ones who lit the match.
Gal Ratner is Chief Architect at Prana Entertainment and the founder of Inverted Software and WhiteStar Labs. He writes about AI, software architecture, and the economics of platform decline at galratner.com.