Topological Data Analysis: Extracting Meaning From Big Data
We live in an era of Big Data.
Businesses collect and analyze clients’ records to drive growth. Healthcare companies use biometrics and stats from sensors to better patient care. Data examination is now a crucial part of development strategy for most firms worldwide.
However, as the world’s digitalization advances and the number of online transactions grows at a frenetic pace, the quantity and complexity of data sets grow, too — exponentially.
At this point, even the most sophisticated firms often find themselves lacking analytical capabilities to process and extract meaning from an overwhelming flow of data they receive.
So what are corporations to do? Is there a way for them to examine data beyond conventional hypothesis-driven analysis?
A group of experts in Topological Data Analysis (a new study practiced and commercialized by Ayasdi) claims to have a solution.
Topology, the discipline from which TDA originates, is a branch of mathematics that concerns itself with measurement and representation of shapes. As part of math, it has been studied for over 250 years. But it’s in the last 20 years, however, that scientists have begun to research applications of topology to various real world problems.
The two main functions of the discipline — measurement and representation of shape — are especially relevant in the context of analyzing big, highly complex, feature-rich data sets.
And thus Topological Data Analysis became a thing.
What do you think of when someone says data?
I’ve always imagined sets of numbers and maybe some mysterious abbreviations piled up in spreadsheets. Or, sometimes, percentages. Or mathematical fractions.
Whichever image it is, the data has never looked simple or understandable in my head. I’ve thought of it as requiring encryption and imagined only trained analysts, people with a firm grip on advanced math concepts, to be able to squeeze any insights out of it.
TDA is set to change the way we perceive data. It claims that data has shape.
Here’s how it works. Suppose there’s a large data set that is divided into groups.
But instead of conventional columns that are used to view data, the groups are represented by nodes, which are connected to one another reflecting the relationships between the groups.
Now, instead of a plethora of figures and columns and rows, which are all unstructured and incomprehensible, we have something more pleasant-looking: a network. Our data now has shape.
And since human visual perception is powerful, it is simple for us to identify features within such a network that correspond to patterns within our data set.
These patterns are precisely what TDA experts are referring to when they say that data has meaning.
