4 Things Every Leader Should Know Before Starting A Big Data Project

We were told that “data is the new oil.” The Internet of Things combined with the ability to store massive amounts of data and powerful new analytical techniques like machine learning would help derive important new insights, automate processes and transform business models. It seemed like a massive opportunity.

Yet Gartner analyst Nick Heudecker‏ estimates as many as 85% of big data projects fail, due to a lack of data skills, poor internal coordination between departments and lack of integration with line managers and staff. Implementing a big data project, it seems, is far more challenging than installing a new email system.

The news isn’t all bad. A survey by Deloitte of “aggressive adopters” of cognitive technologies found 76% believe that they will “substantially transform” their companies within the next three years. So clearly, while big data and cognitive technologies are no panacea, they can deliver value, if pursued wisely. Here’s how you can keep your data project from going off the rails.

In the Deloitte survey, one major concern that even leaders who are successful implementing cognitive technologies have is integration into existing processes and systems, with nearly half saying that it is a top challenge. However, Roman Stanek, CEO at GoodData, a firm that specializes in data analytics and intelligence, believes this is in large part a problem of conception.

“The first question you have to ask is what business outcome you are trying to drive,” he told me. “All too often, projects start by trying to implement a particular technical approach and not surprisingly, front line managers and employees don’t find it useful. There’s no real adoption and no ROI.”

One company that has been successful with this approach is CAVA, a fast growing restaurant chain similar to Chipotle but focused on healthy Mediterranean cuisine. It been almost fanatical about making sure line managers are involved with data projects from the start, in many cases initiating them, rather than just having new systems thrust upon them by IT staff.

“What’s been essential is that our data science team has built genuine partnerships with our other teams with the intention of extending the capabilities of every facet of our business, rather than trying to replace human talents with a set of algorithms,” CAVA CEO Brett Schulman told me.

While many worry that cognitive technologies will take human jobs, David Autor, an economist at MIT sees the the primary shift as one of between routine and nonroutine work. In other words, artificial intelligence is quickly automating routine cognitive processes much as industrial era machines automated physical work.

One of the most basic mistakes many firms make is to try to use cognitive technologies to replace humans to save costs rather than to augment and empower them to improve performance and deliver added value. This not only kills employee morale and slows adoption, it usually delivers worse results.

Stanek advises his customers to start with automating the most tedious tasks first.