If there is an “original sin” to intellectual culture of the last 500 years, it is the Big Data fallacy—the idea that if we can just gather up enough raw data, we can finally understand everything we need to know about the world.
Is someone not happy? All you have to do is develop a better set of metrics for personality tests. Is consciousness not revealing its secrets? All you need to do is model the brain more perfectly. Are people behaving in unpredictable ways? You just need more information on their habits and shopping patterns.
More data, the theory goes, always provides more clarity.
Practitioners of the human sciences, like existential-humanists, have always knows this isn’t true: Nietzsche’s perspectivism, the idea that there is no privileged or uniquely correct view of reality, presaged Einstein’s relativity.
What you know depends on who you are and where you stand. Self-knowledge—of exactly the kind that is least quantifiable—not only can’t be factored out, but it is also essential.
One doesn’t need hard data to prove this—indeed, that’s exactly the point—but it’s there, for those who don’t believe their mothers love them until they can synthesize it in a lab.
I’ve written in this space before about experiments showing that that participants’ math education doesn’t make them less likely to make math errors supporting arguments that are aligned with their political beliefs—in fact, the more educated one is, the more likely one is to twist the math to fit the belief. Now new research, recently profiled in The New Yorker, furthers this point: studies out of Dartmouth and Bristol demonstrate that providing people with factual information that contradicts their already established beliefs is actually counter-productive to changing their minds, if the beliefs in question are important to them.
The only time this research has found that facts and data are effective (to some extent) in changing minds on important issues is when—get this—the subjects are first helped to feel competent and positive about themselves in the area up for debate.
One wonders what such a dialogue would look like: if New Atheists would be willing to try to talk “sense” into religious believers by first affirming them, or vice versa. It’s hard to imagine proponents of climate change willing to affirm and support deniers of climate change before going over the facts again, and vice versa.
In the current socio-intellectual climate it would never happen. They want to win the argument, but for too many people in the vanguard of each side, winning by first affirming the rationality and legitimacy of the people on the other side is far too high a price to pay.
And why is that? It’s because we (whichever side we’re on) are as afflicted by this phenomenon as the people we disagree with. We assume we’re right—we know we’re right—but we’re as unwilling to consider contradictory evidence as anyone else. Which makes it very easy to think that anyone who disagrees with us must be an idiot, or a liar, or a coward …
And from there you’re an easy step away from the condition of The Polarized Mind, as Kirk Schneider has described it in his book of the same name.
Crunch the numbers all you want, quantify as hard as you can, this is not rationality.
This is, of course, a much older problem than “big data”—but big data is uniquely incapable of addressing it. The idea that if you just crunch the numbers hard enough, the truth will emerge is precisely the kind of mind-set that keeps us from seeing clearly. What we need are self-awareness, humility, and empathy.
For that, we need to focus on the intangibles. Without them, big data can be counter-productive data.