Killer Correlators

My twin brother made an interesting observation the other day about the field of education. While kids are in school, they read plenty of textbooks. They read scores of primary source documents like The Federalist Papers. And they read dozens of novels. But why aren't they reading books like Malcolm Gladwell's Outliers? These books teach the very skills that we say we are looking for: critical reasoning, creative thinking, scientific observation and curiosity about the way the world works. Now I see that there's another one of these books out now, called Catching Fire, that gets a great review in the New York Times. The thesis: the invention of cooking is the reason humans developed a host of other modern physical traits. Fascinating, fact-based, paradigm-shifting, thought-provoking. And just one in a long line of a new field of study: Correlology.

From Freakonomics, to Guns, Germs and Steel, to Moneyball, to The Tipping Point, to The Black Swan, a new way of thinking about the world is ascendant. While the books are firmly in the realm of social science, they have nothing to do with socialism vs. capitalism, atheism vs. religiosity or Freud vs. Jung. All of these books pivot off of one major point: common human assumptions about the way things work and relate to each other -- about the correlation between cause A and effect B -- have been overlooked. You know you're reading one of these books if you see a bunch of sentences start with the phrase "Recent research shows that while other factors play a role, there is one factor that truly explains phenomenon X."

In a way, these authors are selling conspiracy theories with science to back them up. They are like conspiracy theories (and just as oddly appealing) because they unveil the secret forces behind the phenomena we see, from the sudden success of the Oakland A's (Moneyball) to the sudden success of a trend (The Tipping Point). But their data, stats and research gives them the appeal of science along with the thrill of secret-sharing. More than anything else, these books are selling the killer correlator. In a very emergent sense, they don't tell you the exact decisions to make -- they tell you what decisions will matter. They tell you the proper metrics to evaluate.

Now, social science obviously wasn't invented in the past few years. But for some reason, the past few years has seen a wave of popular books in this field of Correlology. Why should this be so?

I think part of the answer is that because of our increasingly hyper-social service economy, understanding social science is taking on increasing importance to increasing numbers of people. Second, our institutions are becoming more transparent and meritocratic. Old methods of running institutions, in which ideology, basic corruption and a good ole' boys' networks pre-determine decisions, are no longer sufficient. In a world of metrics, where thousands or millions of people are armed with data and evaluating your decisions, getting it right is more important than ever. In a world of complete mobility, where free agents or employees or stock holders or customers are ready and willing to pick up and go at the drop of a data point, figuring out how humans work is more key to success than it has ever been.

I personally find these kinds of books fascinating. And while I haven't been following the New York Times bestseller list for decades, I have noticed that these books do seem to have come into their own in recent years. While I offered a few suggestions above, I can't quite wrap my mind around why these books have soared in the popular imagination. Please leave your suggestions in the comments section -- I have a feeling that this is a subject I'll be posting about again.