Glossary / AI
Large language model
An AI trained on vast amounts of text that generates language by predicting the next word. The core of ChatGPT, Claude and similar systems.
The transformer architecture, published in 2017, made it efficient to model relationships between distant words in a text[1]. Scaled up to hundreds of billions of parameters — adjustable numbers — it becomes a large language model. GPT-3 in 2020 showed that sheer scale lets a model handle tasks from just a few examples[2].
Its inner workings are not a program anyone wrote but an enormous set of learned numbers. Even its developers cannot say why it gives a particular answer; interpretability research tries to read it from the inside.
Related terms
- Mechanistic interpretabilityThe research field that tries to read an AI's internal computation part by part and explain, as a mechanism, why it produced an answer.
- HallucinationWhen an AI generates plausible-sounding content that is not grounded in fact or contradicts its source.
- NeuronThe smallest unit of computation in a neural network. It weights and sums its inputs and produces an output depending on the result.
Sources
- A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention Is All You Need,” Advances in Neural Information Processing Systems 30 (arXiv:1706.03762) (2017).
- T. B. Brown, B. Mann, N. Ryder, et al., “Language Models are Few-Shot Learners,” Advances in Neural Information Processing Systems 33 (arXiv:2005.14165) (2020).