Why Big Data Is Not Truth
The word “data” connotes fixed numbers inside hard grids of information, and as a result, it is easily mistaken for fact. But including bad product introductions and wars, we have many examples of bad data causing big mistakes.
Big Data raises bigger issues. The term suggests assembling many facts to create greater, previously unseen truths. It suggests the certainty of math.
That promise of certainty has been a hallmark of the technology industry for decades. With Big Data, however, there are even more hazards, some human and some inherent in the technology.
Kate Crawford, a researcher at Microsoft Research, calls the problem “Big Data fundamentalism — the idea with larger data sets, we get closer to objective truth.” Speaking at aconference in Berkeley, Calif., on Thursday, she identified what she calls “six myths of Big Data.”
In 1997, there was a paper that discussed the difficulty of visualizing Big Data, and in 1999, a paper that discussed the problems of gaining insight from the numbers in Big Data. That indicates that two prominent issues today in Big Data, display and insight, had been around for awhile.
“But now it’s reaching us in new ways,” because of the scale and prevalence of Big Data, Ms. Crawford said. That also means it is a widespread social phenomenon, like mobile phones were in the 1990s, that “generates a lot of comment, and then disappears into the background, as something that’s just part of life.”
Over 20 million Twitter messages about Hurricane Sandy were posted last year. That may seem sufficient for a picture of whom the storm affected. However, the 16 percent of Americans on Twitter tend to be younger, more urban and more affluent than the norm. “Very few tweets came out of Breezy Point, or the Rockaways,” Ms. Crawford said. “These were very privileged urban stories.” And some people, privileged or otherwise, put information like their home addresses on Twitter in an effort to seek aid. That sensitive information is still out there, even though the threat is gone.
That means that most data sets, particularly where people are concerned, need references to the context in which they were created.
“Big Data is neither color blind nor gender blind,” Ms. Crawford said. “We can see how it is used in marketing to segment people.” Facebook timelines, stripped of data like names, can still be used to determine a person’s ethnicity with 95 percent accuracy, she said.
