A logical calculus of nervous activity

In December 1943 a neurophysiologist and a homeless teenage logician published a paper that reduced the nerve cell to a switch. Warren McCulloch was 45 and ran a laboratory in Chicago. Walter Pitts was 20, had no degree, and had taught himself logic from Russell and Whitehead. Their model neuron takes a set of inputs, adds them up, and fires only if the total crosses a threshold. Nothing about chemistry, nothing about time, just a sum and a step.

The point of the simplification was what it allowed them to prove. Wired together, these units compute the AND, OR and NOT of logic, and from those any finite logical expression. Give a network loops and it can represent anything a Turing machine can compute. The brain, they argued, could be understood as a machine that evaluates propositions.

Almost everything on this site descends from that idea. Von Neumann cited the paper in his 1945 design for the stored-program computer. Rosenblatt's perceptron is a McCulloch–Pitts unit with adjustable weights. Every deep network today is a stack of units that sum inputs and pass the total through a nonlinearity, which is the 1943 neuron with a softer edge. The threshold became a sigmoid, then a rectifier, but the shape of the thing has not changed in eight decades.