AI Timeline, 1676–2024: From the Chain Rule to ChatGPT

The chain rule — how to trace influence through nested calculations

Reasoning backwards from results to causes — Bayes's essay

Least squares — minimise the sum of squared misses

Gradient descent — take small steps down the steepest slope

Counting 20,000 letters of Pushkin — predicting what comes next from what came before

A row of tiny magnets pointing up or down — an equation that later became a model of memory

What does it mean to compute? — defining one machine that can compute anything

Writing the nerve cell as a component of logic

Measuring information with numbers — and producing English-like text from probabilities alone

Learning is a change in the strength of connections — Hebb's hypothesis

Replacing 'Can machines think?' with a question that can be tested

Even with noisy measurements, you arrive if you keep shortening your step

The perceptron — a machine that adjusts its connections every time it is wrong

The layers in between can be learned by going downhill too — Shun'ichi Amari's theory

Perceptrons — a proof that some problems cannot be solved with one layer

A method for studying rounding error turned out to be backpropagation itself

The neocognitron — a layered network that recognises a pattern even when it shifts

A network that holds memories as valleys — a physicist's way of seeing it

The Boltzmann machine — teaching hidden units the features of the data

Backpropagation spreads — the layers in between start building features of their own

Learning without waiting for the outcome — using the gap between one prediction and the next

One layer in between is enough to get as close as you like to any function

Support-vector machines — a method guaranteed to find the best answer becomes the mainstream

LSTM — carrying a clue from the distant past without losing it

Representing words as lists of numbers and having a network predict the next word

Preparing a deep network one layer at a time before training it

Attention — choosing where to look in the source for each word of the translation

Turn an image gradually into static and learn to run it backwards — the prototype of diffusion models

ResNet — shortcuts that skip layers take networks past a hundred layers

The Transformer — stop reading in order and build with attention alone

Lay a foundation on vast amounts of text first, then adapt to the task