What is Small and Wide Data, and How is Big Data Different?

Whether reading Greek philosophy or listening to songs on the radio, we’re often reminded that the only thing that stays the same is that everything changes. In the realm of research and analytics, one of the most important changes currently influencing individuals, corporations and even politics is a shift in focus away from the concept and capabilities of big data.

If the era of big data is ending, what is taking the place of such a powerful and influential practice? With computing power increasing exponentially and advances across the information technology space, one might think that the next evolution would be bigGER Data!

Somewhat surprisingly, industry leaders believe that the opposite is true. According to a May 2021 research report from a leading technology advisory firm Gartner, small and wide data is what the top academics and analysts focus on for the future. Gartner experts say that 70% of Organizations Will Shift Their Focus From Big to Small and Wide Data By 2025.

In this article, we will first introduce the concept of small and wide data, together with some real-life examples of how practitioners are using them in the market. Let’s also dig into how small and wide data differs from big data and why this change is expected to remain long into the future.

It’s clear that this trend away from big data and towards small and wide data is more and more critical for companies to understand, and we’ll give you the top reasons why.

Sometimes it’s easiest to start with what is already familiar — big data. Ever since computer hardware and software became sufficiently powerful to deal with unimaginably huge data sets, scientists and mathematicians have run meaningful, academic statistical analyses, and investors have embraced “more, faster and better quality” data.

As the internet age continued to produce vast reams of data, such as the seemingly infinite amounts gathered and stored by technology companies operating search engines (like Google), social media platforms (like Facebook) and computerized financial exchanges, the appeal of big data grew and grew.

Key characteristics of big data, such as volume, frequency, and variety, lead to its use being somewhat limited to building bigger picture ideas. It’s great for visualizing a particular market trend or understanding the distribution pattern of its components. In other words, big data is an excellent way to figure out whether you are looking at a tree or a building.

Suppose you are an AI developer. In that case, big data can show you the percentage of companies using AI In their market tech stack or the ratio of companies using it for generating leads. It becomes quite a valuable piece of information to understand the level to which the market is interested in AI-based software, doesn’t it?

In contrast, small and wide data is better at picking out more specific information and distinct insights from individual data components and drawing valuable comparisons. To use our tree/building example, it’s more about looking at the leaves on the tree or focusing on a particular room in a building as a means of understanding not only what the thing is, but how it works and why it’s there in the first place. But this is just a vague explanation to get us rolling. Let’s look deeper.

Wide data allows the analyst to examine and combine a variety of small and large, unstructured and structured data. In comparison, small data is focused on applying analytical techniques that look for useful information within small, individual sets of data.

Specifically, wide data is all about tying together disparate data sources across a wide range of sources to come up with meaningful analysis. Consider this example from a systematic trading strategy. Based on data gathered from asset price movements, asset valuation factors etc., a systematic investment strategy can be created. That is, just a few simple factors across a wide range of data.