Teach the machine.

Each tray holds labelled examples; each example is two numbers, its position on the plate. Model A is logistic regression: three weights, and the wire it draws is always straight. Model B is a network with two hidden layers of eight units (97 weights), trained by the same rule, and its wire can bend. “Run 50 steps” shows the model every example fifty times, nudging the weights after each one to reduce the error. That is all that is learned: the weights. The features, the shape of the model and the objective were chosen before the first example arrived.

The ring cannot be separated by a straight wire; switch to B and it can. Then add the held-out set: examples drawn from a slightly different world. Some land on the wrong side of a wire that fit its training examples perfectly. A model that fits has been fitted, not proved right. Everything runs in your browser, seeded, so the same clicks draw the same wire twice. A small real classifier trained here in the browser; not the capacity or architecture of every system in this chapter.