Random forests

Leo Breiman was a statistician who had spent thirteen years as a consultant before returning to Berkeley, and his methods had the flavour of someone who needed things to work on real data. Random forests, published in October 2001, grow many decision trees, each from a bootstrap sample of the data and each choosing its splits from a random subset of the features, and take a vote. The randomness makes the trees disagree, and averaging over disagreement cancels their errors.

The method was nearly impossible to misuse. It needed no scaling of the inputs, no tuning to speak of, handled thousands of features, and reported which ones mattered. For a decade it was the first thing a practitioner tried on tabular data and often the last, and in fields such as genomics and remote sensing it still is.

Breiman also wrote, the same year, the essay "Statistical Modeling: The Two Cultures", in which he argued that his discipline had wasted itself on models that explained data instead of models that predicted it. He died in 2005, before the second culture won so completely that the argument became hard to remember.