data science, including analytics, big data, and artificial intelligence, is no longer a novel concept. Nor is the important foundation of high-quality data. Both have contributed to impressive business successes — particularly among digital natives — yet overall progress among established companies has been painfully slow. Not only is the failure rate high, but companies have also proved unable to leverage successes in one part of the business to reap benefits in other areas. Too often, progress depends on a single leader, and it slows dramatically or reverses when that individual departs the company. In addition, companies are not seizing the strategic potential in their data. We’d estimate that less than 5% of companies use their data and data science to gain an effective competitive edge.
Over the years, we have worked with dozens of companies on their data journeys, advising them on the approaches, techniques, and organizational changes needed to succeed with data, including quality, data science, and AI. From our perspective, these are the two biggest mistakes organizations make:
Although the details at each company differ, seeing data too narrowly — as the province of IT or the data science organization, not of the entire business — is a recurring theme. This causes companies to overlook the transformative potential in data and therefore underinvest in the organizational, process, and strategic changes cited above. Similarly, they blame technology for their quality woes and failures to capitalize on data, when the real problem is poor management.
We’ve all observed how companies behave when they are truly serious about something — how the goal changes from incremental progress to rapid transformation; how they muster both breadth and depth of resources; how they align and train people; how they communicate new values and new ways of working; and how senior leaders drive the effort. Indeed, it almost seems as if companies go overboard when they are truly serious about something. Amazon’s Project D initiative to develop the Echo/Alexa smart speaker is a great illustration of that seriousness, with hundreds of employees, several startup acquisitions, heavy CEO involvement, and no expense spared. DBS Bank’s journey to being named World’s Best Digital Bank by Euromoney is another good example. The company’s CEO, Piyush Gupta, said the following upon receiving that award in 2018:
The contrast with most companies’ data programs is stark — one can only conclude that many are not yet serious about data and data science. For those only beginning to explore data, this may be understandable. But, if you’ve been at it for three years or more, it is time to either get serious in addressing mistakes or invest your resources elsewhere — and expect to lose out to competitors.
The obvious approach to addressing these mistakes is to identify wasted resources and reallocate them to more productive uses of data. This is no small task. While there may be budget items and people assigned to support analytics, AI, architecture, monetization, and so on, there are no budgets and people assigned to waste time and money on bad data. Rather, this is hidden away in day-in, day-out work — the salesperson who corrects errors in data received from marketing, the data scientist who spends 80% of his or her time wrangling data, the finance team that spends three-quarters of its time reconciling reports, the decision maker who doesn’t believe the numbers and instructs his or her staff to validate them, and so forth. Indeed, almost all work is plagued by bad data.
The secret to wasting less time and money involves changing one’s approach from the current “buyer/user beware” mentality, where everyone is left on their own to deal with bad data, to creating data correctly — at the source. This works because finding and eliminating a single root cause can prevent thousands of future errors and eliminate the need to correct them downstream.
