Tay

Tay was released on Twitter on 23 March 2016 as an experiment in conversational understanding, modelled on a nineteen-year-old American and designed to learn from the people who talked to it. A coordinated group found within hours that "repeat after me" was honoured, and that the bot's learning did the rest. By the evening it was posting praise of Hitler and abuse of feminists and the Jewish people. Microsoft took it down after sixteen hours and apologised two days later. A Chinese predecessor, Xiaoice, had run for two years without incident.

The episode is small and its lessons are large. A system that learns from its users learns what its worst users teach it; a model's behaviour is a product of its data; and the public will test any released system to destruction on the first day. All three lessons were applied, expensively, by the laboratories that shipped chatbots six years later, with reinforcement learning from human feedback and red teams whose job was to be the Twitter users of 2016 before release.

Tay is also the first widely covered case of an AI system being withdrawn for its behaviour, a pattern that recurred with Galactica in 2022, Bing's chat mode in 2023 and Gemini's image generator in 2024.