Nate Silver’s improbably entertaining book on statistics and forecasting, “The Signal and the Noise: Why So Many Predictions Fail — but Some Don’t,” is Northwestern University’s One Book One Northwestern all-campus read for the 2016-17 academic year.
Silver, the founder and editor-in-chief of FiveThirtyEight.com, will deliver a keynote address and sign books at Northwestern on Oct. 6, less than a month before the 2016 presidential election. “The Signal and the Noise” will be given to all incoming and first-year students at Northwestern and will anchor related films, lectures and other programming throughout the year.
“Nate Silver’s work bridges the universe of people who deal with quantitation and those who deal with life,” said One Book faculty chair Stephen Carr, professor of materials science and engineering and of chemical and biological engineering at the McCormick School of Engineering.
“He makes the world of data science accessible, but he won’t tell you how he does it. He leaves that to the mystique of his craft,” Carr said.
In “The Signal and the Noise” Silver breezily investigates how predictions are made in a wide range of fields, including chess, baseball, weather forecasting, politics and earthquake analysis. He offers hopeful examples (baseball and weather predictions), but weighs the progress against a series of catastrophes that no one saw coming, such as the September 11 attacks, the global financial crisis and the earthquake and tsunami in Fukushima.
“There are entire disciplines in which predictions have been failing, often at great cost to society,” Silver wrote.
Reviewers have called Silver everything from “the Kurt Cobain of statistics” to “a sort of Zen master to American election watchers.”
The Economist wrote: “In the spirit of Nassim Nicholas Taleb’s widely-read ‘The Black Swan,’ Mr. Silver asserts that humans are overconfident in their predictive abilities, that they struggle to think in probabilistic terms and build models that do not allow for uncertainty.”
The choice is a natural one for Northwestern given the school’s investments in interdisciplinary work, data science and quantitative analysis, said Eugene Lowe Jr., chair of the One Book selection committee. “As a community we’re trying to enhance understanding and appreciation for how one thinks about statistics and mathematics across different domains,” Lowe said. “Silver has helped a wide range of people understand difficult statistical material.”
Silver’s investigation into the noisy world of data began at his website FiveThirtyEight, which refers to the number of votes in the Electoral College. FiveThirtyEight uses statistical analysis — hard numbers — to help readers understand politics, polling, sports, science and culture.
In 2008, the FiveThirtyEight forecasts correctly predicted the winner of the presidential contests in 49 of 50 states as well as the winner of all 35 U.S. Senate races. In 2012, Silver called the correct winner in all 50 states, prompting “The Daily Show’s former host Jon Stewart to call him the “lord and god of the algorithm.”
Silver called the outcome “a stroke of luck.” Though fortuitous, it “contributed to the perception that statisticians are soothsayers – only using computers rather than crystal balls,” he wrote. “This is a wrongheaded and rather dangerous idea. American presidential elections are an exception to the rule and far more often, we overrate our ability to predict the world around us.”
The book makes both philosophical and technical recommendations. “Once we’re getting the big stuff right — coming to a better understanding of probability and uncertainty; learning to recognize our biases; appreciating the value of diversity, incentives and experimentation — we’ll have the luxury of worrying about the finer points of technique,” he wrote.
Silver argues that we have more information than we know what to do with, but relatively little of it is useful. “We think we want information when we really want knowledge,” he wrote.
“The signal is the truth. The noise is what distracts us from the truth.”
The One Book One Northwestern Program is sponsored by the Office of the President. “The Signal and the Noise” was chosen from more than 70 nominations and chosen by the One Book selection committee.