Google reveals the TPU

At its I/O conference on 18 May 2016, Google mentioned that its data centres had for more than a year been running a chip it had designed itself, the Tensor Processing Unit, built to do one thing: the eight-bit matrix multiplications of neural-network inference. It had powered the search ranking model RankBrain and, in March, AlphaGo's match against Lee Sedol. The 2017 paper by Norman Jouppi's team put the numbers on it, fifteen to thirty times the performance of a contemporary GPU or CPU on Google's workloads, at thirty to eighty times the performance per watt.

The disclosure changed the hardware market. Until then, NVIDIA's graphics processors, designed for games and adapted for learning, were the only serious option. The TPU showed that the arithmetic of deep learning was regular enough to deserve its own silicon, and that a company running models at Google's scale would save enough power to justify building it. Amazon, Microsoft, Meta, Tesla and a dozen startups followed with chips of their own.

The second-generation TPU of 2017 could train as well as infer, and the pods of thousands of them trained BERT, the transformer's successors and Gemini. The largest language models in the world have been trained about equally on Google's chips and NVIDIA's.