Monday, 12 November 2012

The Signal And The Noise by Nate Silver

From The Week of November 5, 2012

Though human progress relies on several critical pillars in order to function, none are more vital to the survival of the species than predicting the future. For it's only through the reasonably accurate anticipation of what's coming that we have any notion of how to act. The world around us is rife with data: dark clouds that presage rain, the approaching roar that announces an oncoming car, the rot that decays our food. If we were suddenly unable to use these data to draw logical conclusions based on past experience, we'd be dead in short order, felled by an inability to understand, to know, to make an educated guess. It's not a quality often valorized; we're rarely stripped of our predictive powers, a reality that leads us to take its virtues for granted. But this blindness has not stopped Mr. Silver from properly identifying it as a key aspect of progress and change. He explains in his captivating treatise.

From meteorology to poker, from elections to personal relationships, our world runs on guesswork. There are simply too many unknowable variables involved in the interaction of an ecosystem with the seven billion souls it shelters. There is no way for us to possess all of the information necessary to understand all. Thus, ever since we've been endowed with the powers of intellect, we've used a clumsy combination of guesswork and past experience to anticipate the future. But this is a deeply flawed model of deciding how to act. For we are, all of us, profoundly ill-equipped to be objective about ourselves and our world. We place far too much emphasis on our own actions, our own positions in the world, for our memories and our predictions to be trustworthy. But what if we could harness the understanding of the masses? What if we could eliminate errors of bias by using the average of the many to correct for the skewed and skewable beliefs of the few? Could we then have a reasonable expectation of a true future?

Perhaps, but correcting for bias is only half of the equation of understanding. The rest is rooted in the reality that, though we cannot know all the variables, we can grasp some of them. We can tease out of our world statistics that reveal some of the laws that underpin our actions in the micro and the universe in the macro. And we can use these statistics, along with what we know of ourselves and our fellows, to build models that can finally begin to conceive of our we act as a whole.

Mr. Silver understands this truth. A self-described math geek and a lover of baseball and poker, his youth was marinated in stats, stats that, if they could only be plugged into a broader matrix, could be made to reveal knowledge of how certain systems function. Armed with such information, one could reasonably maximize one's opportunity to win at all manner of endeavors, from trivial games to consequential bargaining sessions. For one would not need to waste time with bad strategies shaped by desperation and ego masquerading as intuition. One could simply possess a superior comprehension of the game and play within that framework, never descending to the muddy realm of unquantifiable speculation. After dabbling in the numbers of baseball and poker, Mr. Silver applied precisely this logic to the prediction of American elections. The result? A nearly flawless scorecard in the last two presidential races, an astounding level of accuracy that flirts with the magical until one absorbs the lessons in his wonderful book.

From sportsbetting to weather prediction, The Signal And The Noise walks readers through the worlds of statistics and of predictive modelling, illuminating the revelations of all manner of far-flung disciplines that all of us rely upon but few of us respect. But though it bestows upon us a welcome understanding of how parts of the world function, its truths are far more profound. For this is nothing less than a tome on how humans think and act, on how we can harness that data to our individual and societal advantage, and how we ought to couple that data with our scientific understanding of the universe to see more clearly where we are going. But in order to do this, we must relinquish our biases, all of them. We must reject the pretty fictions we cling to in order to ground ourselves in this shared reality. For to do anything less is to give up the exercise entirely. It's to reject the objective world for a subjective one, one in which there are no universal truths, one in which story supplants hard data. And though this might have been acceptable a century ago, when we lacked the tools to gather these data, we no longer have any excuses. Understanding is out there. We've merely to embrace it in all its definable glory.

For the degree to which it educates and philosophizes, illuminates and ruminates, this is one of the year's best reads and arguably one of the humblest books I've ever read. As profound as it is exquisite.. (5/5 Stars)

No comments:

Post a Comment