Self-organising maps

Teuvo Kohonen's self-organising map, published in 1982, learns with no teacher. Its neurons sit on a grid. Each input is compared with every neuron's weights, the closest neuron wins, and it and its neighbours on the grid are pulled a little towards the input. Repeat over many inputs and the grid folds itself into the shape of the data, so that nearby neurons respond to similar things and the map preserves neighbourhoods. The inspiration was the cortex, where adjacent patches of tissue respond to adjacent patches of skin or retina.

The map was one of the few network methods used widely in industry through the 1980s and 90s, for speech, process monitoring and, in Kohonen's own group, for organising a million Usenet posts by topic. It is the ancestor of the dimensionality-reduction pictures, t-SNE and UMAP, that researchers use to look at the embeddings of modern models.

It is also an early example of a lesson the field kept relearning: that structure in data can be found without labels, and that the hard part of learning is not the answers but the representation. Word2vec in 2013 and the self-supervised methods of the 2020s are that lesson at scale.