Caught in the crossfire of the distillation wars

Negative Zero builds software with an agentic harness in which three separated roles work in fresh contexts: a planner scopes each sprint, a generator builds it, and an evaluator tries to prove the result wrong. The roles never share a conversation; they hand work to each other through files on disk, and six of our projects now run this way.

The file that matters here is log.md, an append-only record of every action, decision and result in a project. We had always thought of it as the audit trail, but it is also an input, because every role reads the end of the log before starting work so that it knows where the project stands.

When the planner stopped starting

In late September a new generation of Anthropic's Claude models shipped, and from then on the planner role could not get started. The generator and the evaluator carried on normally, but every planner spawn on the new generation was stopped by Anthropic's safety classifier before it had written anything, each time reporting the category reasoning_extraction. When we next put the harness to work, two of our projects logged more than a dozen of these refusals, and a sprint in one of them sat blocked while we looked for the cause.

The whole episode fitted inside a single day. The first refusal came at 19:40 on a Sunday evening, a spec written on the previous model generation kept that sprint moving by 21:05, and the fix that removed the cause took about four hours on the Monday morning, after which every project planned normally on the new generation.

What the classifier is there to stop

The category in those refusals describes a request that asks the model to write its internal reasoning into its reply. Claude can reason through a problem before it answers, and Anthropic's documentation says that the raw chain of thought is never returned to the caller in full. The classifier that kept stopping our planner exists to protect that boundary, and Anthropic takes it seriously enough to bill a refusal that arrives before any output.

That classifier was new to the models our harness runs on. Anthropic's refusal documentation lists Claude Sonnet 5.5 among the models that carry these safety classifiers and does not list Claude Sonnet 5, and its launch post for Claude Opus 5.5 describes it as the first Opus model to ship with the class of distillation safeguards first introduced on Claude Fable 5.1. Our planner had moved from Sonnet 5 to Sonnet 5.5 at the end of September, and none of the models it had run on before carried this safeguard, which is why every earlier upgrade to a more capable generation had gone through without breaking a role. The same documentation notes that this category has no recommended fallback model and advises changing the prompt rather than retrying, which is why moving between the new models made no difference.

What the boundary protects against is distillation, which means training a smaller model on a larger model's outputs. Geoffrey Hinton, Oriol Vinyals and Jeff Dean named the technique in 2015, and laboratories use it on their own models all the time. It becomes a problem when the outputs belong to somebody else, because a competitor with API access can send a frontier model millions of chosen prompts and train on the answers, acquiring for the price of the API calls much of a capability that took years to build. Reasoning is the most valuable material for this, since a transcript of how a model reached an answer teaches far more than the answer alone.

Anthropic has been publishing what it finds. In February it reported that DeepSeek, Moonshot AI and MiniMax had run more than 16 million exchanges with Claude through roughly 24,000 fraudulent accounts, including prompts designed to make Claude write out its chain of thought, and in September it named Alibaba, Zhipu, Xiaomi and SenseTime as well, describing an Alibaba campaign of more than 151 million exchanges between May and July. These are accusations from the company whose model was queried rather than findings tested in court, and OpenAI has made similar claims about its own models. On 12 October, Reuters reported Anthropic as saying that its safeguards had cut the reasoning extracted by distillers to between ten and a hundred times less than before, that it had banned tens of thousands of accounts, and that the system runs with a false positive rate below 0.1%.

None of this was on our minds when the refusals began, because our planner was doing ordinary engineering work on our own code. The background explains why a classifier tuned against campaigns of hundreds of millions of requests will sometimes stop a legitimate customer, but it did not tell us why it had singled out one role in our harness and left the other two alone, and that took us until the next morning to establish.

Following the wrong leads

Our first suspicion fell on the prompt that starts each role, but rewriting it made no difference to the next attempt. Since the same task still completed on the previous model generation, it was tempting to conclude that the new generation was broken, and for a while we pinned the planner back a generation so that work could continue.

The role's charter, the standing instructions it reads before acting, turned out to be part of the answer. The planner's charter was the only one asking the role to narrate its own deliberation, whereas the generator and evaluator record commands, outputs and evidence. We rewrote it to ask for an audit record instead, setting out the sources the planner read and placing each item of scope beside the clause that authorises it. To test the change we ran the old and new charters side by side on the same model, with the same prompt and repository, in the same minute, giving each run its own isolated copy. The old charter was refused on both of its attempts while the new one produced a complete specification on all three of its attempts, and so we shipped it.

Within a few hours the planner was refused again in a live project, with the new charter already on disk, and this time the refusal arrived at the very first step, before the planner had even opened its charter.

What was actually happening

A charter cannot explain a refusal that happens before it is read, and that observation moved our attention away from what we were asking the agent to do and towards what we were giving it to read.

Each refusal until then had been written up carefully in log.md, with the error message and the wording we suspected quoted in full so that whoever came next could follow the reasoning. Because the planner reads the end of that log before it starts, every new planner had been handed a detailed account of the previous refusal, including the very wording we believed had caused it.

The clearest evidence came from a project that had never had a refusal of its own. Its only incident text was a pair of maintenance entries we had added to every project that evening while rolling out the charter fix. With those two entries in its log the planner was refused, and when we removed them and changed nothing else it wrote a full specification. Every other project we tested behaved the same way: a project that had been refused on both of its attempts succeeded on both after the entries were removed, and the project whose sprint had been blocked produced an 836-line specification once roughly 29 KB of incident entries had been taken out of its 1.7 MB log.

The new model generation had never been broken; it was simply more sensitive than its predecessor to text that our own harness had been writing about itself.

It also explains how a safeguard that Anthropic puts at a false positive rate below 0.1% could stop one customer more than a dozen times in a row. A rate like that is an average across everything the classifier sees, and our requests were not independent of one another: every planner we started read the same entries at the end of the same log, so whatever tipped the classifier once tipped it again on almost every attempt. A rare false positive becomes a dependable one when an agent keeps handing the classifier the same input, and that is what our harness had been doing.

How we changed the harness

Incident write-ups now live in a directory called harness/incidents/, which sits outside the set of files any role is told to read, so the detail is preserved without any agent ever loading it. When a role is refused, the log gets a single factual line naming the role, the model, the category and the request identifier, and the analysis goes into the incident file.

We also made the rule enforceable rather than leaving it as guidance. Across our projects the rules that have held are the ones backed by an automated check, while those that existed only as prose have tended to drift, so the harness check that already validates the log's format and its append-only history now warns whenever a refusal entry grows beyond a few hundred characters. In the same spirit, whenever a new model generation ships we now run each role once on a scratch copy of a project and record the exact model identifiers that passed, and the check warns if a role later runs on an identifier nobody has tested.

Removing the incident entries from six logs meant making a deliberate, one-off exception to our own append-only rule. Every byte we removed was copied into the incident file and verified there first, and each log records the exception. All six projects now plan on the current model generation without any pins.

What we would pass on

The most useful thing we took from this is that an agent's audit trail is part of its input. Anything an agent reads shapes what it does, including records written with people in mind, so it is worth deciding deliberately what each agent reads and keeping forensic detail somewhere it does not. It also pays to ask an agent for the record you actually need: our planner had been asked for a narrative when the harness needed evidence, and correcting that made the harness better whatever the classifier did. Every wrong turn we took traced back to an experiment that was missing something, whether a shared model, a shared input, a single variable or an unchanged control, and the rules that lasted were the ones we turned into checks.

We raised the refusals with Anthropic as a false positive. None of this suggests the classifier was wrong to be cautious, and our changes simply stop the harness from looking like something it is not.

Seeing it sooner

We found this problem by reading logs, which is slower than it needs to be. Every refusal came back with a category and a request identifier, and the pattern was there in the usage data before anyone opened a log: one role, on one model generation, stopped at its first step again and again while the other roles carried on normally.

That kind of pattern is what our AI Assurance service is built to catch. It monitors your Claude estate continuously from usage metadata, never conversation content, and routes unusual activity to the Negative Zero Agent for triage, so a run of refusals from one agent reaches your team in Slack or your SIEM as an alert that carries the action to resolve it, rather than surfacing later as a stalled sprint. We run it on our own Claude estate first, which is where lessons like this one come from.

If you are building agentic delivery into your own engineering, our Forward Deployed Consulting team works inside your repositories on exactly this kind of harness, and AI Assurance keeps watch over it once it is running.