# Backpropagation

Date: 9 October 1986 · Era: II · Connection · Category: theory · Significance: 5/5
People: David Rumelhart, Geoffrey Hinton, Ronald Williams
Organisations: University of California San Diego, Carnegie Mellon University

Canonical: https://shapeofintelligence.com/timeline/1986-backpropagation/ · Dataset and text CC BY 4.0

> Rumelhart, Hinton and Williams show that multi-layer networks can learn internal representations by propagating errors backwards; Perceptrons is answered.

## What had to happen

7 events back to 1943, oldest first. Direct influences are marked *direct*.

- [1943 · A logical calculus of nervous activity](https://shapeofintelligence.com/md/timeline/1943-mcculloch-pitts-neuron/)
- [1949 · Cells that fire together wire together](https://shapeofintelligence.com/md/timeline/1949-hebb-organization-of-behavior/)
- [1958 · The perceptron learns](https://shapeofintelligence.com/md/timeline/1958-perceptron/)
- [1960 · ADALINE and the least-mean-squares rule](https://shapeofintelligence.com/md/timeline/1960-adaline-lms/) *direct*
- [1969 · Perceptrons](https://shapeofintelligence.com/md/timeline/1969-perceptrons-book/) *direct*
- [1970 · Reverse-mode automatic differentiation](https://shapeofintelligence.com/md/timeline/1970-linnainmaa-backprop/) *direct*
- [1974 · Werbos applies backpropagation to neural networks](https://shapeofintelligence.com/md/timeline/1974-werbos-backprop/) *direct*

The paper is three pages long and appeared in Nature on 9 October 1986. Its claim is in the title: a network with hidden layers can learn representations, and the way to train it is to compute the error at the output and pass it backwards, layer by layer, using the chain rule, so that every weight in the network receives its share of the blame. The gradient that results is the direction to move, and moving a little at a time is gradient descent. David Rumelhart had worked it out in 1982; Geoffrey Hinton and Ronald Williams helped him make it convincing.

The mathematics was not new. Linnainmaa had the algorithm in 1970 and Werbos had applied it to networks in 1974. What the 1986 paper did was demonstrate, on problems people cared about, that the hidden units learned something interpretable, that the network solved XOR and family-tree relationships and the encoder problems Minsky and Papert had used as evidence of hopelessness. It arrived in the two-volume Parallel Distributed Processing books the same year, and a generation of researchers learned it from there.

Everything in the deep-learning era is trained this way. Convolutional networks, LSTMs, transformers, diffusion models: the architectures differ, and the loss functions differ, but the weights are always set by backpropagation and some variant of gradient descent. Rumelhart died in 2011. Hinton's 2024 Nobel citation begins with this paper.

## What it led to

96 events downstream, through 2026.

Built on it directly:

- [1987 · NETtalk learns to read aloud](https://shapeofintelligence.com/md/timeline/1987-nettalk/)
- [1987 · The first NIPS conference](https://shapeofintelligence.com/md/timeline/1987-nips-conference/)
- [1989 · ALVINN drives a van with a neural network](https://shapeofintelligence.com/md/timeline/1989-alvinn/)
- [1989 · The universal approximation theorem](https://shapeofintelligence.com/md/timeline/1989-universal-approximation/)
- [1989 · LeNet reads handwritten postcodes](https://shapeofintelligence.com/md/timeline/1989-lenet/)
- [1990 · Finding structure in time](https://shapeofintelligence.com/md/timeline/1990-elman-network/)
- [1991 · The vanishing gradient problem](https://shapeofintelligence.com/md/timeline/1991-vanishing-gradient/)
- [1992 · TD-Gammon reaches world-class backgammon](https://shapeofintelligence.com/md/timeline/1992-td-gammon/)
- [2003 · A neural probabilistic language model](https://shapeofintelligence.com/md/timeline/2003-neural-language-model/)
- [2006 · Deep belief networks and the word 'deep'](https://shapeofintelligence.com/md/timeline/2006-deep-belief-nets/)
- [2014 · Adam](https://shapeofintelligence.com/md/timeline/2014-adam/)
- [2019 · The Turing Award goes to deep learning](https://shapeofintelligence.com/md/timeline/2019-turing-award/)
- [2024 · The Nobel Prizes go to neural networks](https://shapeofintelligence.com/md/timeline/2024-nobel-prizes/)

And, through them, by era:

- III · Statistics and data: 7 events
- IV · Deep learning: 17 events
- V · Transformers: 17 events
- VI · Everyone: 28 events
- VII · Agents: 14 events

## Sources

- [Learning representations by back-propagating errors (Nature, 9 October 1986)](https://doi.org/10.1038/323533a0) — paper
- [Learning representations by back-propagating errors (Nature article page)](https://www.nature.com/articles/323533a0) — paper
- [Scientific background to the 2024 Nobel Prize in Physics](https://www.nobelprize.org/prizes/physics/2024/advanced-information/) — article

---

The Shape of Intelligence by Jamie McKaye (https://jamiemckaye.com) · Markdown mirror of https://shapeofintelligence.com/timeline/1986-backpropagation/

Before: [1986 · ID3 and decision-tree learning](https://shapeofintelligence.com/md/timeline/1986-id3-decision-trees/) · After: [1987 · The Lisp machine market collapses](https://shapeofintelligence.com/md/timeline/1987-ai-hardware-crash/)
