Goodfire Researcher Says AI Neurons Are Far Easier to Read, Rejecting His Own Theory

Nick Cammarata opened a grid of cat faces and car fronts he had looked at hundreds of times. The Goodfire researcher was revisiting an old puzzle: why would the same part of an AI image model respond to such different things? This time he asked what shape they shared. Within seconds, he saw a V. Checking the connections brought him to pointed flames and water jets from a fireboat.

He had co-written a 2020 paper featuring the example with his mentor Chris Olah; together they named the theory explaining it, superposition, an explanation he now says was wrong.

In an essay published October 15, 2026, Cammarata contrasts the OpenAI project he started in 2023, which could explain about 1.7% of the neurons it studied, with his preliminary audits of an openly available language model, Qwen3-32B. Roughly nine in ten neuron sides examined now had one main job, though these are different models and tools and the figure is preliminary. His findings challenge an explanation that helped steer researchers toward building separate models to interpret the originals.

“Instead, I spent most of a decade figuring out how to look at a model rather than looking at it,” Cammarata writes.

What a neuron is

A neural network is a computer system made of connected units that learn from examples. An artificial neuron is a unit within that system whose numerical response changes with its input. It might respond strongly to a pointed shape. Connections carry that response onward, helping other neurons reach a decision.

The research field called mechanistic interpretability tries to trace how those internal parts produce an answer. Researchers examine connected groups of neurons, called circuits, and test their explanations by changing or removing parts.

A neuron that appears to do several unrelated jobs is called polysemantic. The cat-and-car example helped persuade researchers that individual neurons might be poor starting points for understanding a model.

The knot theory

The leading explanation was superposition. In Cammarata’s account, when a network runs short of neurons, it starts putting unrelated concepts together because they rarely interfere. A detector for floppy-eared dogs might also hold information about Latin poetry.

That would make the network a kind of knot. Studying a neuron directly would reveal an arbitrary combination rather than a useful explanation.

Researchers therefore trained another model to separate the mixtures. A common tool, a sparse autoencoder, learns to reconstruct the original model’s activity using a small selection from a larger set of possible patterns. Researchers study those extracted patterns as candidates for the underlying concepts.

Cammarata now thinks the apparent mixtures usually have a connection researchers missed. As he puts it, “If the network isn't a knot, untangling it can make it look like one.”

The old image models

The V offered a testable explanation. Flames and fox faces fit it too. Round O shapes pushed the neuron’s response strongly negative while pointed shapes pushed it positive, two opposing sides called its poles.

Another circuit had appeared to send car information into unrelated dog neurons. Cammarata and Olah interpreted that as concealed storage. On reexamination, the grouping followed pose: dogs and cars facing the camera in similar ways.

Their picture-making tool contributed to the mistake. Feature visualization generates images intended to provoke a strong response from a neuron. In InceptionV1, an image-recognition model from 2014, it often found dog images because the training data contained many dogs. The resulting pictures hid shared patterns.

In Cammarata’s retraining tests, the old network design caused its later layers to overflow, while weak tools made their activity look still less coherent. A newer network design of the same size, trained on the same images, produced understandable neurons. Design changes intended to help networks learn better had the unintended effect of making them easier to read.

How meanings grow

A coworker found a neuron that an automated tool had labeled a “polysemantic comma neuron.” Closer examination showed that it tracked Colorado, including phone numbers and local slang. The commas marked places where the model stored a running summary of the Colorado context. The team checked that guess by turning up the neuron’s response and seeing what the model wrote: it wanted to go hiking.

Cammarata calls his alternative explanation recursive generalization: a neuron learns a pattern, then sprouts offshoots the way a tree branch sprouts sub-branches.

A language-model neuron responded to the letters “iy,” then expanded to “diy,” then to “do it yourself.” The final phrase contains no “iy.” Following how the neuron developed made the relationship visible.

AI agents mapped neurons and circuits and tested explanations, at a scale no human team could. Cammarata worked with Goodfire colleague Curt Tigges on the language-model experiments.

The numbers

The language-model audits counted poles, meaning the strongly positive or strongly negative side of a neuron’s response. A main job could include small related extensions. Cammarata cautions that the estimate depends on where researchers set thresholds for weak responses and what they count as a single job. It is a broad indication, not a precise score.

His image-model tests described in the October essay offer a ladder: by his recollection, about 5% of the old model’s last-layer neurons were understandable with the tools of the era. Better tools brought that to about 50% on the same 2014 network, retraining to about 70%, and a modern network of the same size to about 99%.

Outside researchers have not yet confirmed the results, the technical companion paper with the language-model experiments has not been released, and Goodfire, an interpretability company, has a stake in the thesis.

Why safety researchers care

Alignment means getting AI to act toward human goals. Today researchers mostly judge a model by what it writes, including the reasoning it shows.

Cammarata hopes reading the system could allow “simple, careful changes,” such as removing a toxic training environment, a set of practice tasks a model learns from. The parts behind deceptive behavior are hard to know “without going in and looking.” He suggests understanding the whole network may be easier than studying only the parts researchers care about most.

The pushback

Neel Nanda leads Google DeepMind’s mechanistic interpretability team, which has already reduced its emphasis on sparse-autoencoder research. “I do think that using neurons is just vastly easier to, for example, do on a frontier model, which is great,” he wrote.

Language models seem to hold far more facts than they have neurons, he argues. He expects weak neuron responses to carry much of that information, which an audit focused on the strongest responses could miss even if nine in ten poles have an identifiable main job. He also doubts that findings from small image models transfer readily to language models: he expects the image models to memorize far less. Being able to explain prominent activity would not give him confidence that nothing important had been missed.

“This is interesting work, but I don't think it's a significant update to the big picture of interpretability,” Nanda wrote on X.

Liv Gorton, an interpretability researcher who applied sparse autoencoders to the old image model’s early layers in 2024, welcomed the prospect of more understandable networks. To her, superposition means a model holding more ideas than it has neurons, so Cammarata’s branching neurons do not contradict it.

Cammarata accepts that models can represent more concepts than they have independent numerical directions. His objection concerns the claim that unrelated concepts are packed together arbitrarily.

Roon, a close friend of Cammarata’s and an OpenAI employee posting under a pseudonym, wrote that the work “may be remembered as a major moment in scientific history.”

What comes next

Cammarata labels his broadest conclusion, that understanding models could prove much easier soon, “Highly speculative.”

His October essay puts confidence in the proposed learning process at “maybe 85% sure.” That is his stated confidence in an explanation, not a measured share of understood neurons.

Cammarata writes, “To my knowledge, no one has found a superposition circuit in a real network since my original one, which was an error.” He thinks “the bar for claiming it should be high.” The model he audited, Qwen3-32B, is openly available, so other researchers can run the same check. The companion paper is still to come.

Uno Researchers Report Up to 3× Faster AI Without Sacrificing Quality

Large language models generated tokens up to 3× faster than their own autoregressive base models in tests reported in a September 3 Uno preprint, while preserving those models’ output distributions. U

The Implicator

Z.ai Says Chinese Chips Matched Nvidia Cost Serving GLM-5.3-Flash

Z.ai said Wednesday that the week-long anonymous preview of GLM-5.3-Flash ran entirely on domestically developed Chinese chips and reached per-token costs comparable to mainstream Nvidia GPUs. A custo

The Implicator

Alibaba Ships Qwen 3.8 27B With Maximum Reasoning Effort Turned On by Default

Alibaba released Qwen3.8-27B with its maximum xhigh reasoning effort enabled by default. A request to draw a pelican riding a bicycle as an SVG used 22,276 reasoning tokens over 21 minutes. Early outs

The Implicator

Marcus Schuler

San Francisco

Editor-in-Chief and founder of Implicator.ai. Former ARD correspondent and senior broadcast journalist with 10+ years covering tech. Writes daily briefings on policy and market developments. Based in San Francisco. E-mail: editor@implicator.ai