Glossary / AI
Sparse autoencoder
An auxiliary neural network used to decompose an AI's internal activity into combinations of a very large number of features, only a few of which are active at once.
It takes the neuron activity of one layer of a model as input, re-expresses it using only a few (sparse) out of tens of thousands to millions of candidates, and is trained to reconstruct the original. Studies in 2023 showed that the resulting candidates tend to correspond to human-understandable concepts[1].
The drawback is cost: a large extra network has to be trained alongside the model being studied. In 2026 work appeared suggesting that in some cases neurons can be read directly, without this conversion, and still be interpreted.
Related terms
- FeatureIn interpretability research, a unit of an AI's internal representation that corresponds to a human-understandable concept, such as the Golden Gate Bridge.
- Mechanistic interpretabilityThe research field that tries to read an AI's internal computation part by part and explain, as a mechanism, why it produced an answer.
- NeuronThe smallest unit of computation in a neural network. It weights and sums its inputs and produces an output depending on the result.