IV · Statistical learning, datasets and computing, 1980s–2010s
The examples
What changed when machines learned from data?
Learning from examples means giving up on writing the rule. You choose a model with adjustable parameters, show it labelled examples, and move the parameters to reduce the error. A perceptron draws a straight boundary; a network with a hidden layer can bend it. Three things decide what gets learned: the examples, the objective the model is judged by, and its capacity to represent a boundary at all. Statistics supplied the method through the 1990s: support vector machines (1995), probabilistic models, and in 1998 the MNIST digit set that became the field's shared examination. Data grew with the web. ImageNet (2009) grew to more than fourteen million labelled images, labelled by people paid per task through Amazon's Mechanical Turk. A model that fits its examples has not been proved right; it has been fitted. What happens on the next example, from a slightly different world, is the question the examples cannot answer.
Teach the machine
The ring after 300 steps of model B, the held-out set added: the wire bends around the inner tray; squares from a slightly different world that land on the wrong side are ringed coral.
Artefacts
Twelve digits from the MNIST test set
1998 · document, image to follow
Twelve handwritten digits from the MNIST test set (LeCun, Cortes and Burges, 1998), the examples a generation of models was examined on. CC BY-SA 3.0. MNIST, Yann LeCun and Corinna Cortes, CC BY-SA 3.0. authentic · MNIST, Yann LeCun and Corinna Cortes, CC BY-SA 3.0 · CC BY-SA 3.0 · source
From the archive
- 1995Support-vector machines
- 1998MNIST and LeNet-5
- 2009ImageNet
Claims in this chapter
ImageNet grew to more than fourteen million labelled images, labelled by people paid per task through Amazon Mechanical Turk.
unverified · ImageNet — About