LeNet reads handwritten postcodes

Yann LeCun arrived at Bell Labs in 1988 from Geoffrey Hinton's group in Toronto and was given a problem the post office cared about: reading the handwritten postcodes on envelopes. His answer, published in Neural Computation in December 1989, combined Fukushima's architecture with Rumelhart's learning rule. Small filters slide across the image and share their weights, so the network looks for the same feature everywhere and has far fewer parameters than a fully connected one; pooling layers make the result tolerant of small shifts; and the whole stack, filters included, is trained end to end by backpropagation.

The network had about 9,700 parameters and was trained on 7,291 digits scanned from envelopes at the Buffalo post office. It made about five percent errors, good enough that by the mid-1990s versions of it were reading a large share of the cheques deposited in American banks.

Within the second winter, this is the quiet work in the cold. LeCun's 1998 paper on LeNet-5 and the MNIST dataset made the design the standard benchmark, and when the same architecture met GPUs and a million images in 2012, it ended the winter that had followed the expert systems. The convolutional network is the one idea from this period that scaled without modification.