Some Forecasts Are All Wet

Nate Silver's new book The Signal and the Noise: why so many predictions fail -- but some don't details his methods of assessing statistical probabilities of future events -- and he comes to the same conclusion as Yogi Berra: "It's tough to make predictions, especially about the future" and while this advice isn't groundbreaking, his treatment of it might be . . . he reminds us that we have a fantastically large amount of information available now, yet the accuracy of many of our predictions don't necessarily reflect this added information-- we still can't sift "the signal" out of the chaos, and so we need to know what "noise" to ignore -- and often the most significant information comes either in the tiny details, at the "more granular" level or in the big story . . . at a largely philosophical level (his explanation on how Standard and Poor's and Moody's blew the CDO risk assessment and contributed to the financial crisis is excellent) and while his explanation of how he built PECOTA -- an algorithm designed predict the success of baseball players over the course of their career -- is engaging and fun, my favorite chapter so far explains the truth about weather forecasting: the National Weather Service does a great job, but Weather.com and your local weatherman have a "wet bias," because the worst thing a weather service can do is NOT predict rain . . . so if there is a 5% or 10% or 15% chance of rain, Weather.com will say that there is a 20% chance of rain -- to avoid the ire of folks who might get rained on when they didn't think that there was a chance in hell it was going to rain . . . and since rain makes such good TV, on your local forecast, if they predict a 100% chance of rain, the rain only occurs 66% of the time . . . it's kind of like setting your clock a few minutes ahead so you're not late for work . . . you're fooling yourself for your own good.