What’s Holding Us Back Now? ‘It’s the Data, Stupid’

The good news is the barrier to entry for data science has lowered dramatically in recent years, thanks to better data science software and cloud computing. The bad news is that getting ahead with big data requires–you guessed it–access to more and better data.

In some ways, the “data is the differentiator” story has not changed. Even when organizations were struggling to get their Hadoop environments up and running 10 years ago and get all of the various software products working together, the goal was always to build a platform to do something creative, fun, or profitable with data.

The difference today is that a lot of the other stuff that previously got in the way of leveraging data–namely, assembling the hardware and software stacks needed to facilitate advanced analytics and training machine learning models–has gotten a lot better.

Thanks to the nearly unlimited compute resources available on public cloud platforms, and the “glut of innovation” that Gartner has identified data science and machine learning applications, the old big data barriers have been torn down. These are heady days for data science and big data practitioners, to be sure.

So now that ready-made data science and advanced analytic platforms that can crunch huge amounts of data are available at our beck and call, what’s holding us back from getting down to business and doing great things with the data? To paraphrase a political consultant, “It’s the data, stupid.”

The amount of data in the world continues to grow at a rapid pace. According to IDC, there was 64.2 zettabytes of data created or replicated in 2020. Over the next five years, IDC projects data to increase at a 23% annual compound growth rate. So there is plenty of data to be had. The big question is how will that data be distributed, and which companies will take advantage of it.

One vendor that’s aiming to get more and better data into the hands of data science teams is Narrative. The New York City company hosts a streaming data platform that connects data buyers with data sellers, enabling companies of all sizes to swing above their (data) weight.

“The tech is there for smaller companies to compete” with the FAANGS of the world, says Nick Jordan, the CEO and founder of Narrative. (FAANG, of course, refers to the tech giants like Facebook, Amazon, Apple, Netflix, and Google [plus Microsoft].) “In order to really compete though, they’ve got to figure out a way to have some semblance of the scale of data that the FAANGs have.”

Narrative’s platform helps automate much of the integration, security, and regulatory work that arises when working in a third-party data marketplace. The company has an equal balance of data buyers and data sellers, Jordan says. It turns out that, when a company starts the process to begin buying third-party data, they often come the realization that their data has value to others, too.

“Our job is to make it so someone who isn’t steeped in this type of technology can do this, and it looks like it’s magic, it’s no longer hard,” Jordan says. “Data used to be the purview of the nerds. And that was great.