Increasingly, external data (alternative data, public data, open data – call it what you want) is being called the “secret sauce” of driving advanced analytics, developing machine learning and AI capabilities, enriching existing models, and delivering unrealized insights to every part of your organization.
The difficulty in connecting to this data is top of mind for many businesses, and the issue of governing its use is crystallizing into a core strategy among organizations that have seen the GDPR’s writing on the wall.
For the moment, put aside the slightly fevered predictions about delivering artificial intelligence across the enterprise with a few datasets and a new CDO (setting up an external data strategy relies as much on culture as it does on data). It’s important to acknowledge that for many organizations, external data should be approached not as a replacement for what’s already working, but as an enhancement to what you’ve got today.
Most organizations use data. Many use it effectively. Regardless of where an organization is on their journey towards being data-driven, the chances are good that data, in some form or another, is already hardwired into their architecture. For a lot of these organizations, the excitement about external data is not that it’s a brand-new resource, but that it has the potential to enrich existing processes to make them better.
For example: as a business owner, I wouldn’t consider replacing census data as a core data element, but if a solution came along that let me layer customer footfall data over that core demographic layer, I’d jump at the opportunity to enrich my existing data.
What you do with the data is up to you. But increased access to better information has never been a bad thing.
For some organizations, however, there’s almost too much potential. The idea that there’s a virtually unlimited supply of valuable data out there that can be applied anywhere to achieve any result leads to a lack of direction and focus. “I want more data” is a great sentiment, but it’s not a great strategy. We call this the Blue Sky Problem: given complete freedom to start using an unlimited resource, it can be difficult to know where to start.
As a rule, start with what you know. Net new data products can take a while to test and build into a production environment. Starting from scratch can mean that you must wait for setup, testing, deployment, and iteration before you’re seeing real results. Long before that happens, the data will be flagged as an unnecessary expense and the project will be abandoned. It takes either a very confident or a very foolish organization to pour money into a project where no one’s thought through how it will be productionalized.
This doesn’t mean you shouldn’t allocate time and resources towards pure data science where teams are encouraged to experiment without a deterministic outcome, but these skunkworks projects should account for maybe 20% of your external data budget and resource allocation. The remaining 80% of your resources should go towards projects where you know that additional or better data can improve an existing process.
For some organizations, this can be a difficult concept to grasp.
