# TD-Gammon reaches world-class backgammon

Date: May 1992 · Era: W2 · The second winter · Category: model · Significance: 4/5
People: Gerald Tesauro
Organisations: IBM

Canonical: https://shapeofintelligence.com/timeline/1992-td-gammon/ · Dataset and text CC BY 4.0

> Gerald Tesauro's network learns backgammon by playing itself with temporal-difference learning and reaches the level of the best humans, changing how they play.

## What had to happen

12 events back to 1943, oldest first. Direct influences are marked *direct*.

- [1943 · A logical calculus of nervous activity](https://shapeofintelligence.com/md/timeline/1943-mcculloch-pitts-neuron/)
- [1948 · A mathematical theory of communication](https://shapeofintelligence.com/md/timeline/1948-shannon-information-theory/)
- [1949 · Cells that fire together wire together](https://shapeofintelligence.com/md/timeline/1949-hebb-organization-of-behavior/)
- [1950 · Programming a computer for playing chess](https://shapeofintelligence.com/md/timeline/1950-shannon-chess/)
- [1958 · The perceptron learns](https://shapeofintelligence.com/md/timeline/1958-perceptron/)
- [1959 · Samuel's checkers program coins 'machine learning'](https://shapeofintelligence.com/md/timeline/1959-samuel-machine-learning/) *direct*
- [1960 · ADALINE and the least-mean-squares rule](https://shapeofintelligence.com/md/timeline/1960-adaline-lms/)
- [1969 · Perceptrons](https://shapeofintelligence.com/md/timeline/1969-perceptrons-book/)
- [1970 · Reverse-mode automatic differentiation](https://shapeofintelligence.com/md/timeline/1970-linnainmaa-backprop/)
- [1974 · Werbos applies backpropagation to neural networks](https://shapeofintelligence.com/md/timeline/1974-werbos-backprop/)
- [1986 · Backpropagation](https://shapeofintelligence.com/md/timeline/1986-backpropagation/) *direct*
- [1988 · Temporal-difference learning](https://shapeofintelligence.com/md/timeline/1988-temporal-difference-learning/) *direct*

TD-Gammon was a network with one hidden layer of forty units, later eighty, that took a description of a backgammon position and predicted who would win. Gerald Tesauro at IBM trained it by having it play itself, hundreds of thousands of games and eventually over a million, adjusting its predictions after every move with Sutton's temporal-difference rule. It was given no strategy and no expert games. By 1992 it played at a strong level; by 1995 it was as good as the best humans in the world, and a few of its opening moves, which contradicted a century of received wisdom, were adopted by them.

It was the first program to reach the top of a serious game by learning rather than by search, and it did so in the middle of the second winter, on hardware that would fit in a wristwatch today. Tesauro's papers were read carefully by a small community and ignored by the mainstream, which regarded backgammon's dice as a special case.

They were the special case that generalised. AlphaGo Zero in 2017 is TD-Gammon's recipe with a deep network and tree search: self-play, a value function learned from outcomes, no human data. Tesauro's forty hidden units turned out to have been the right idea waiting for thirty years of Moore's law.

## What it led to

45 events downstream, through 2026.

Built on it directly:

- [2010 · DeepMind is founded](https://shapeofintelligence.com/md/timeline/2010-deepmind-founded/)
- [2013 · Deep Q-networks play Atari](https://shapeofintelligence.com/md/timeline/2013-dqn/)
- [2016 · AlphaGo beats Lee Sedol](https://shapeofintelligence.com/md/timeline/2016-alphago/)
- [2017 · AlphaGo Zero learns from nothing](https://shapeofintelligence.com/md/timeline/2017-alphago-zero/)

And, through them, by era:

- IV · Deep learning: 2 events
- V · Transformers: 6 events
- VI · Everyone: 19 events
- VII · Agents: 14 events

## Sources

- [Practical issues in temporal difference learning (Machine Learning, 1992)](https://doi.org/10.1007/BF00992697) — paper
- [Temporal difference learning and TD-Gammon (Communications of the ACM, 1995)](https://doi.org/10.1145/203330.203343) — paper

---

The Shape of Intelligence by Jamie McKaye (https://jamiemckaye.com) · Markdown mirror of https://shapeofintelligence.com/timeline/1992-td-gammon/

Before: [1991 · The first Loebner Prize](https://shapeofintelligence.com/md/timeline/1991-loebner-prize/) · After: [1992 · The Fifth Generation project ends](https://shapeofintelligence.com/md/timeline/1992-fifth-generation-ends/)
