Nate Silver Book Review: The Signal and the Noise Offers a Sober Analysis of Prediction

Here are Nate Silver’s last few weeks: he successfully predicted the winner of all 50 states in the presidential election, became the subject of an internet meme (Drunk Nate Silver), and began being approached by television producers to star in a Hollywood project.

With seemingly everything coming up Nate Silver, PolicyMic decided to look back at the New York Times blogger’s breakout book, The Signal and the Noise: Why So Many Predictions Fail — but Some Don't.

Silver was something of a sensation within internet poker and baseball circles long before the New York Times purchased his blog FiveThirtyEight, a political prediction model brought under The Grey Lady’s banner before the 2008 presidential election. But Silver's following grew at the conclusion of that cycle when he correctly predicted 49 of 50 presidential states, a number he rather remarkably improved upon in the just concluded-race between Barack Obama and Mitt Romney.

In between the election cycles, Silver went to work on his book examining the predictive failures of everything from the Wall Street crash that triggered the Great Recession to TV pundits. He interviewed meteorologists, sports bettors, and the operators of supercomputers. The question at the center of the interviews and Silver’s meticulous research is this: Why in an era of “Big Data,” with “2.5 quintillion bytes” generated daily do so many predictions so badly miss the mark? According to Silver, it’s because predictors are poor at separating the signal from the noise. “The signal is the truth,” he writes, adding, “the noise is the distraction.”

Silver’s analysis is easily digested and the book makes for a breezy, informative and often fun read (no easy feat for a book dealing in Bayesian reasoning). He’s at his best when writing about baseball and the revolutionary system, PECOTA, he developed during slow hours at the consulting job in which he worked following college. Silver explains how the system forecasted the success of baseball players, how it stacked up against professional scouts and how baseball teams have come to use the “statheads” and “jocks” to complement one another. It’s in the chapter on baseball that Silver’s conclusion emerges clearest: there’s no silver-bullet magic to prediction. Instead, predictors must draw careful conclusions based on available data and adjust when new information provides clear “signals.”

Interesting did-you-know moments you’ll want to share with anyone in earshot abound in this book, like the plausibility of using a mathematical proof to predict an affair and the existence of a bias toward predicting rain among local TV meteorologists.

One conclusion that can be made at the end of this more than 500-page book requires almost none of Silver’s nuance. To wit, Silver’s status as pop culture’s star statistician will endure for the foreseeable future.