5 Keys to Getting Your Big Data Transformation Back on Track

We’re almost a month into the Major League Baseball season, and every year there’s at least one fan base that gets caught in the following trap.
Despite modest expectations, their team gets off to a fast start. Career .250 hitters look like Ted Williams. Historically light hitting outfielders are suddenly on pace to hit 70 home runs.  Pitchers that once appeared to serve up mere batting practice are suddenly unhittable.

And then reality and regression to the mean sets in. Despite the hopes for a miraculous off-season transformation, the team turns out to be what it is. They sink in the standings and corresponding disappointment sets in.
Many Big Data initiatives we have seen with our global enterprise customers have fallen into similar doldrums. They gained quick traction and visibility with a high impact business use case, but now that efforts are focused on scaling, operationalizing, and demonstrating ongoing success, efforts are stalling. ‘Regression to the mean’ is occurring in the enterprise sense – the common barriers we’ve all seen due to organizational complexity, cultural resistance to change etc. have brought initiatives back to reality.

Based on the initial excitement and promise of Big Data, many of our customers have developed an overall framework for how technology, data people and process need to come together to support the global enterprise. While versions certainly differ, these visions typically look something like this:

These frameworks typically have four major components and associated capabilities required for success:

Deliver Business Value – the first is an understanding of specifically where and how advanced analytics can drive business value and competitive advantage across a variety of use cases and users, including IoT and intelligent applications, data scientists and analysts, as well as regular business users.

Enable Big Data as a Service – to drive the use case ‘pipeline’, data consumers must have self-serve access to the right data and tools. This means rapid service provisioning using a self-serve operating model with appropriate user access controls, similar to what enterprises have enabled with IaaS cloud.

Provide Effective Workspaces – in addition to access to tools and data through self-serve catalogs, organizations also must provide shared workspaces for access to work with target data sets. These environments need to optimize compute resources whether they are deployed internally, in the public cloud or in a hybrid model.

Optimize Data Capture and Storage – finally organizations must have an optimized environment for data ingestion and storage that is optimized for both performance and cost.

So given this relatively clear vision for analytics success, why are so many Big Data initiatives stalling, and what can be done to back on the path to success despite the early excitement? Here’s our quick take based on clients we’ve worked with:
No perceived ROI / Business Value – while the first low hanging fruit use cases were easy, most organizations lack the data science talent to know what problems can be solved with advanced analytics and how. Without this understanding, developing a value-based pipeline is difficult, making it difficult to link Big Data to business value and ROI, let alone assess the overall value of data to the enterprise.

Recommendation: Develop an explicit plan and roadmap for building data science skills and capability, as well as a framework and approach for building a use case pipeline.

Provision time for new services and environments – given the demand for speed and agility, users cannot wait for months to have tools or environments to be made available, whether it be Data Scientists or Business Analysts. This is about more than just providing self-serve catalogs and provisioning, it’s about making sure an overall operating model, including processes and roles, are in place to support it. Far too often this component is overlooked.