When an engineer asks why Claude feels better to work with than the alternatives, they usually expect an answer about size. More parameters, more data, more compute. Size is part of the story — but if it were the whole story, the rankings would reshuffle every time someone trained a larger model, and they don’t. The Claude line has held a lead in agentic coding and long-horizon work across multiple generations, through releases where competitors shipped models that were, on paper, larger or trained on more tokens. Something other than raw scale is doing the work.
There is no single thing. Anthropic models are good because of three layers of engineering that compound: how the model is trained, how it is served, and how it is run. Most public discussion stops at the first layer. The third — the harness that wraps the model in production — is where a surprising amount of the perceived quality actually lives, and it is the layer with the most to teach anyone running models on their own hardware.
Layer one: how the model is trained
Anthropic’s alignment approach, Constitutional AI, replaces a large chunk of human labeling with a loop in which the model critiques and revises its own answers against a written set of principles — a constitution — and then generates its own preference data that drives reinforcement learning, a method known as RLAIF. The practical consequence isn’t just cheaper alignment. It is a model that engages with hard questions instead of stonewalling. The original goal was a model that could be harmless without being evasive, one that explains its objection rather than refusing flatly. Anyone who has watched a competing model dodge a perfectly reasonable question knows how much that matters in daily use.
The second piece is character training, introduced with Claude 3 — the first time Anthropic added an explicit character pass to the post-training that turns a text predictor into an assistant. The goal was richer traits: curiosity, open-mindedness, a willingness to disagree. Amanda Askell, who works on Claude’s character, has described the target as a well-liked traveler who adapts to the people and customs around them without pandering. That phrase carries a real engineering decision underneath it. Anthropic deliberately chose not to optimize Claude to maximize engagement, not to flatter you into another message, on the theory that a good colleague tells you what you need to hear rather than what keeps you hooked. In a market drifting toward engagement-maximizing chatbots, that restraint is something you feel.
There is a more recent and more telling finding. In research published in 2026, Anthropic reported that training on demonstrations of good behavior is often not enough. Their most effective interventions went deeper — teaching the model why one action was better than another, and training on richer descriptions of its overall character. Teaching the principle beat teaching the example. They also found something almost mundane that working engineers should sit with: improving the quality and diversity of the training data produced consistent, surprising gains, and even including tool definitions in the data, whether or not the tools were ever used, made the model better. The lesson is that a frontier model is not one breakthrough. It is a thousand small decisions about data quality, each one compounding on the last.
Layer two: how the model is served
This is where the story of how a model is run begins, before we even reach agents. A model sitting behind an API is only as good as the infrastructure delivering it, and Anthropic has spent real engineering on that delivery.
The clearest example is prompt caching. In an agent loop or a long-context task, the same system prompt, the same tool schemas, and the same conversation prefix get sent on every single turn. Without caching, you pay full price — in money and in latency — to reprocess tokens the model already saw thirty seconds ago. Anthropic’s caching reuses the computed state for a matching prefix, dropping the cost of those cached tokens to roughly a tenth of the base input price and cutting time-to-first-token on long prompts substantially, with reported latency reductions of up to eighty-five percent. The catch is that it is prefix-based: the static content has to come first, before the dynamic parts, and the cache breakpoints have to be placed correctly. That ordering discipline is itself a kind of engineering the platform pushes you toward, and it changes how you architect a system. Caching stops being a developer convenience and becomes infrastructure you plan around the same way you plan batching or concurrency.
The second piece is efficiency as a first-class control. Recent Claude models expose an effort parameter that lets you dial reasoning depth up or down per request, trading latency against thoroughness. The payoff shows up in the numbers. Anthropic reported that Opus 4.5 at medium effort matched the previous Sonnet’s best score on the standard software-engineering benchmark while using around seventy-six percent fewer output tokens. Fewer tokens for the same result is not a vanity metric. Output tokens are the expensive part of inference, and a model that reaches the same answer with a quarter of the generation is cheaper and faster in production. Being good is partly about being economical, and that economy is engineered in.
Layer three: how the model is run — the harness
Here is the part the industry under-discusses, and the part that matters most for where this piece is going. The same model can be brilliant or mediocre depending entirely on the scaffolding around it — the harness that decides what enters the context window on each turn and what the model is allowed to do. Anthropic calls the discipline context engineering, and it is the natural successor to prompt engineering.
The reason it matters is a property of every transformer model, not just Claude. As the context window fills, a model’s ability to accurately recall information from it degrades — a phenomenon researchers call context rot. It comes from the architecture itself: attention scales with the square of the number of tokens, so as context grows the model’s attention is stretched thinner across more and more pairwise relationships, and models have simply seen fewer very-long sequences during training. The result is not a hard cliff but a gradient. The model stays capable but gets less precise the more you cram in. Treating the context window as if it were free is therefore a mistake. It is a finite resource with diminishing returns, and the whole game is finding the smallest set of high-signal tokens that gets the job done.
Anthropic’s harnesses, with Claude Code as the canonical example, manage that resource aggressively, and the techniques are the real product. Rather than loading everything up front, the harness retrieves context just in time, holding lightweight references like file paths and using tools like grep and glob to pull in only what a given step needs, the way a developer navigates a codebase instead of memorizing it. When a conversation approaches the window limit, compaction summarizes the history into a high-fidelity digest and starts fresh, preserving the architectural decisions and open bugs while discarding redundant tool output. The lightest-touch version of that is simply clearing stale tool results: once a tool has run deep in the history, the raw output rarely needs to sit in context forever, and Anthropic shipped exactly that as a platform feature. For longer work, the agent writes structured notes to a file outside the context window and reads them back later, giving it persistent memory across resets. And for the hardest problems, sub-agents each explore with a clean context and return only a compressed summary — tens of thousands of tokens of exploration distilled into one or two thousand tokens handed back to the lead agent.
None of that is the model. All of it determines how good the model appears to be. The clearest evidence sits in Anthropic’s own benchmark reporting. When they measured a competitor’s coding score, they noted it came from a different harness and hosting environment and could not be reproduced on their side — and that simply doubling the resource limits in their own evaluation harness cut infrastructure-related errors to under one percent. Change the harness, change the score, with the weights held perfectly constant. The harness is not packaging around the product. It is half the product.
It is worth keeping this grounded, because the anti-hype point cuts both ways. On the polished public benchmarks, the Claude line now sits high. Opus 4.5 was the first model to clear eighty percent on SWE-bench Verified, and the line has since climbed well into the eighties on the public leaderboards. But the same models drop to the low twenties on harder, private, previously unseen codebases, and every frontier model falls the same way. The lead is real and it is consistent, but it is a lead in a hard, unsolved domain, not a victory lap. That distinction is exactly the kind of thing a good harness either exposes or papers over.
What this means for Ollama and local-model harnesses
Now the part that should land for anyone running models on their own machine. The open-weight models of 2026 are very good. The gap between a frontier release and a capable open-weight model has narrowed to something like a few months, and a well-chosen thirty-billion-parameter model is no longer a toy. So why does the same class of model feel so much weaker when you run it locally than it does inside a hosted product?
The real answer is that tools like Ollama have brilliantly solved the wrong half of the problem. Ollama made local inference trivial — pull a model, run it, switch models, expose an endpoint — and that is a real achievement that put capable models on millions of machines. But running the weights and exposing a chat endpoint is the 2023 conception of running a model. It is the serving layer and nothing above it. To Ollama’s credit, it has started moving up the stack, adding an agent loop, native tools for web search and fetching, human-in-the-loop approval, and an Anthropic-compatible message format. That is the right direction. It is just early, and it is thin next to what a frontier harness actually does.
Two concrete gaps make the point. First, the serving itself quietly degrades the model. Ollama distributes most models at four-bit quantization by default, and many users never realize the model they are running is a compressed derivative with measurably less reasoning depth than the full-precision weights — the command that lists your installed models does not even show the quantization level. A user compares a four-bit local model against a full-precision hosted one, concludes that open weights are weak, and never learns that part of what they measured was the compression. The serving layer is making a silent quality decision on the user’s behalf.
Second, and far more important, there is almost no context engineering. The local harness will happily let the context window fill until the model is rotting on its own history, with no compaction, no tool-result clearing, no just-in-time retrieval, no structured note-taking, and no sub-agents. There is no prefix caching reusing the system prompt across turns, so every turn reprocesses the entire prompt and history from scratch — which is slow, and on local hardware slowness is the entire user experience. The model is being asked to perform while carrying every disadvantage a frontier harness was specifically built to remove.
Here is the advice, stated plainly. The way to make a small local model competitive is not to wait for a bigger small model. It is to build the harness. The quant-versus-size tradeoff that local users already understand — that a smaller model at higher precision often beats a larger model crushed down to two bits — is the whole argument in miniature: how you run a model matters as much as how big it is. A thirty-billion-parameter model with real context management, prefix caching across turns, first-class structured tool use, compaction, persistent file-based memory, and an agent loop as disciplined as Claude Code’s would be a dramatically more useful system than the same model behind a bare chat endpoint. The parameters were never the bottleneck. Everything wrapped around them was.
So the call to Ollama and every harness like it is simple. Stop treating running a local model as an inference problem and start treating it as a context-engineering problem. Ship compaction. Ship tool-result clearing. Ship prefix caching. Ship just-in-time retrieval, persistent file-based memory, and a real sub-agent pattern. The frontier labs have already published how most of it works; the techniques are not secret. The models on local hardware are already smaller, and that is fine — they can compete. They just need to be run like someone meant it.
Gal Ratner is Chief Architect at Prana Entertainment and the founder and CTO of Inverted Software and WhiteStar Labs, a Las Vegas enterprise software and AI consultancy. He has spent nearly thirty years shipping production systems on the Microsoft and .NET stack for companies including Microsoft, Sony, Rockstar Games, 2K Games and Best Buy. His current work is agentic AI in production: MCP servers, the Microsoft Agent Framework, SQL Server 2025 vector search, and diagnostic tooling like PLogger. He writes about enterprise architecture, agentic AI, and the business of technology as "The .NET AI Guy That Ships," with a practitioner-first, ships-or-it-didn't-happen bias. He was a finalist for the Los Angeles Business Journal's CTO of the Year.