There are some things that are so big that they have implications for everyone, whether we want them to or not. Big data is one of those concepts, and is completely transforming the way we do business and is impacting most other parts of our lives.
It’s such an important idea that everyone from your grandma to your CEO needs to have a basic understanding of what it is and why it’s important.
“Big Data” means different things to different people and there isn’t, and probably never will be, a commonly agreed upon definition out there. But the phenomenon is real and it is producing benefits in so many different areas, so it makes sense for all of us to have a working understanding of the concept.
So here’s my quick and dirty definition:
The basic idea behind the phrase 'Big data' is that everything we do is increasingly leaving a digital trace (or data), which we (and others) can use and analyse. Big Data therefore refers to that data being collected and our ability to make use of it.
I don’t love the term “big data” for a lot of reasons, but it seems we’re stuck with it. It’s basically a ‘stupid’ term for a very real phenomenon – the datafication of our world and our increasing ability to analyze data in a way that was never possible before.
Of course, data collection itself isn’t new. We as humans have been collecting and storing data since as far back as 18,000 BCE. What’s new are the recent technological advances in chip and sensor technology, the Internet, cloud computing, and our ability to store and analyze data that have changed the quantityof data we can collect.
Things that have been a part of everyday life for decades — shopping, listening to music, taking pictures, talking on the phone — now happen more and more wholly or in part in the digital realm, and therefore leave a trail of data.
The other big change is in the kind of data we can analyze. It used to be that data fit neatly into tables and spreadsheets, things like sales figures and wholesale prices and the number of customers that came through the door.
Now data analysts can also look at “unstructured” data like photos, tweets, emails, voice recordings and sensor data to find patterns.
As with any leap forward in innovation, the tool can be used for good or nefarious purposes.
