Glossary / AI
Neuron
The smallest unit of computation in a neural network. It weights and sums its inputs and produces an output depending on the result.
In 1943 Warren McCulloch and Walter Pitts described a nerve cell mathematically as a simple element that fires when the sum of its inputs exceeds a threshold, and showed that networks of them can perform logic[1]. Today's artificial neurons descend from that idea.
A large language model has millions to hundreds of millions of neurons. One neuron does not necessarily correspond to one concept — several are usually mixed together — so interpretability research has generally decomposed them into another unit, the feature.
Related terms
- FeatureIn interpretability research, a unit of an AI's internal representation that corresponds to a human-understandable concept, such as the Golden Gate Bridge.
- Large language modelAn AI trained on vast amounts of text that generates language by predicting the next word. The core of ChatGPT, Claude and similar systems.
- Sparse autoencoderAn auxiliary neural network used to decompose an AI's internal activity into combinations of a very large number of features, only a few of which are active at once.