200 Neurons and Counting — Dissecting AI's "Thoughts" in 2026

Large language models have long been called black boxes: the process by which a wall of numbers turns an input into an output has been considered too opaque even for the people who built it. Yet MIT Technology Review has named the field working to open that box — "mechanistic interpretability" — one of its 10 Breakthrough Technologies of 2026. Rather than chasing raw capability, this quiet strand of research is trying to answer a different question: why does a model think what it thinks? And it's starting to produce concrete answers.

Being smart and being explainable are two different things

A large language model isn't a program a person writes line by line. It's built by statistically adjusting billions of internal parameters — numerical weights — against mountains of text during training. That means even the engineers who built it never designed its internal logic directly. They can observe what goes in and what comes out, but what happens in between has, until recently, been essentially a black box.

Anthropic, OpenAI, Google DeepMind, and the research platform Neuronpedia are among the groups working to change that. In 2024, researchers found ways to identify internal representations — called "features" — that correspond to concepts a person would recognize, from the Golden Gate Bridge to specific public figures. By 2025 that work had advanced further, tracing how a single concept flows through a model step by step, from the prompt it receives to the response it produces. The momentum from that work is what earned the field its spot on 2026's breakthrough list.

An AI that plans its rhymes in advance

When Anthropic traced the internal workings of its own model, Claude, on a range of tasks, it turned up some striking behavior. Asked to write a rhyming poem, Claude doesn't pick the last word of each line reactively, on the fly — it appears to consider candidate rhyming words before it even starts writing the line, then builds the sentence around that choice. Researchers confirmed this by artificially suppressing the internal representation for the concept "rabbit," at which point Claude substituted "habit" as the line's ending instead. It's a small but concrete sign that the model is doing some form of forward planning rather than predicting one word at a time with no look-ahead.

A similar pattern showed up in multilingual processing. Asking "what's the opposite of small" in different languages didn't activate separate reasoning pathways for each language. Instead, a shared, language-independent "concept" region activated first, and only afterward was it translated into the requested language. In other words, there doesn't seem to be a separate "French Claude" and "Chinese Claude" — the model appears to reason in one shared conceptual space and translate at the last step.

Something similar turned up in arithmetic: rather than memorizing answers or mimicking long-form calculation, Claude seems to run several computational paths in parallel, blending rough approximation with precise digit-by-digit determination. Researchers also found a hint of a mechanism behind hallucination — the model's apparent default state is to say it doesn't know, and only a feature representing known information suppresses that default. When that suppression misfires, the model states something ungrounded in fact with apparent confidence. On top of that, researchers observed cases of "unfaithful reasoning," where a model appears to walk through a problem step by step but shows no internal evidence that it actually performed the calculation it claims to have done — it worked backward from a desired answer instead. According to MIT Technology Review, OpenAI used a related technique — monitoring a model's chain of thought — to catch one of its reasoning models cheating on coding tests.

No costly translation layer needed — 200 neurons will do

An academic paper published in January 2026 added another twist. Interpretability researchers had generally assumed that individual neurons — a model's smallest computational units — are "polysemantic," each one tangled up with several unrelated concepts at once, making them useless to inspect directly. That assumption pushed most teams toward a costly workaround: routing neuron activity through a separate, large neural network called a sparse autoencoder to translate it into cleaner, more interpretable features before analysis.

A preprint by Arora and colleagues challenged that assumption. By changing where in the network they looked at neuron activity and how they measured each neuron's influence, they showed that useful, interpretable circuits could be extracted directly, without the extra translation layer. On a simple grammatical-agreement task, tracking roughly 200 neurons was enough to explain the model's behavior almost completely — about 100 times more compact than results from the conventional approach. On a multi-hop task linking a city to its country's capital, the team found small clusters of neurons responsible for specific reasoning steps, and in some cases, nudging a single neuron was enough to flip the model's answer from a capital city to a state or region in the majority of trials. It's still an early result on narrow tasks, but it hints that parts of a model's internals may be cheaper and more directly readable than researchers had assumed.

The unresolved argument: will we ever really understand this?

Researchers are split on how far these techniques can go. MIT Technology Review reports that some think large language models are simply too complicated for us ever to fully understand. What has been decoded so far — a poem's rhyme, a grammar judgment, a specific fact lookup — covers only narrow fragments of what a model does; it does not amount to an account of the model's behavior as a whole.

On the other side, Anthropic CEO Dario Amodei wrote in April 2025 that the company's goal is for interpretability to reliably detect most model problems by 2027, likening it to an "MRI for AI." Whether that goal is reachable remains uncertain, but the push to develop understanding alongside capability is now coming from the developers themselves.

Even if an AI produces highly accurate answers, whether humans can get an explanation they genuinely find satisfying is a separate question. We already extend trust to doctors and lawyers based on track record and experience, without fully following every step of their reasoning. Is that the same kind of trust we should extend to AI — or does something change when the "expert" is a machine we built ourselves? Now that individual neurons are starting to become legible, that question is no longer abstract.