AlexNet wins ImageNet

The results of the 2012 ImageNet challenge were released on 30 September. The second-placed entry, from the University of Tokyo, had a top-five error rate of 26.2 percent, a typical increment on the previous year. The winner, a network called SuperVision from the University of Toronto, had 15.3 percent. Nothing in the history of the benchmark, or of the field, had moved that far in one step, and the vision community, which had mostly regarded neural networks as a relic, changed its mind within a year.

The network was Alex Krizhevsky's, with Ilya Sutskever and their supervisor Geoffrey Hinton. It was LeNet's design, eight layers deep, with 60 million parameters, rectified units, dropout, and data augmentation, trained for six days on two NVIDIA GTX 580 gaming cards in Krizhevsky's bedroom using CUDA code he had written himself. Every ingredient existed already; the combination, at that scale, had never been tried.

Within two years every entry in the challenge was a deep convolutional network, error rates fell below human level by 2015, and Google, Facebook, Microsoft and Baidu had bought or built deep-learning groups. Hinton, Krizhevsky and Sutskever's small company was acquired by Google in 2013. If the deep-learning era has a birthday, it is this one.