Former OpenAI Researcher Finds 90% of AI Neuron Functions Explainable

On Oct. 15, Nick Cammarata, a former OpenAI researcher now at Goodfire, published findings suggesting large neural networks may be more interpretable than previously thought, arguing that many apparent signs of chaotic feature overlap stem from analytical limitations rather than intrinsic model structure. In a long essay titled Neural Networks Are Already Clean, Cammarata said that in a preliminary review of Qwen3-32B, about 90 percent of neuron response directions had a clear dominant function, roughly 7 percent encoded two closely related functions, and only 2 percent to 4 percent lacked an identifiable pattern. Similar trends were observed in trillion-parameter models such as Kimi K3. The result challenges the influential superposition hypothesis, which holds that finite neurons must encode many unrelated concepts, prompting heavy use of sparse autoencoders by Anthropic, Google DeepMind and others to decompose mixed activations. Cammarata illustrated the risk of misreading neurons with the classic InceptionV1 neuron reacting to both cat faces and car fronts. Re-examining its behavior and connections, he found it primarily detects a sharp, outward V-shaped geometric pattern also present in fox faces, flames or water jets, rather than unrelated semantic categories. He also compared InceptionV1 with the modern ConvNeXt architecture, finding that ConvNeXt's final layer had about 99 percent neurons with clear functions, and described extreme neuron-compression experiments supporting a process he calls recursive generalization. A key innovation is the use of AI agents to automate mechanistic interpretability: they observe activations, form hypotheses, inspect circuit connections and design verification tests. In a blind evaluation, one agent produced explanations that another independent agent used to generate new test inputs. Across 200 neurons, 76 percent of predicted positive samples strongly activated the target neurons, while false activations from negatives were only 11 percent. The team also reviewed 100,000 attribution graphs to trace information paths. If replicated, the work could shift AI safety research from building elaborate feature-extraction tools toward directly analyzing neurons and circuits with automated agents, potentially helping identify errors, deceptive behavior and training changes. However, the findings are not peer-reviewed, use different models and standards from OpenAI's 2023 GPT-4-to-GPT-2 study that explained only about 1.7 percent of neurons under loose criteria, and do not prove superposition is wrong or solve full-model mechanistic understanding.