Executive Survey: Big Data Has Been a Big Success

With every new technology and management trend, it’s important to examine whether it’s working after a few years. Some, like business process reengineering and Bitcoin, didn’t have a successful first few years. Others, like smartphones and ERP, have been successful enough to prosper for years to come.
The fifth annual Big Data Executive Survey from NewVantage Partners found that most big data initiatives have been successful. Big data was defined broadly for the survey as approaches that enable organizations to analyze any and all data with greater speed and agility. Datasets may be large or small, legacy sources or new sources, structured or unstructured. In the survey, which involved business and technology leaders from 50 leading companies, 48.4% of the respondents said that their firms have achieved “measurable results” from their big data investments. A higher percentage, 80.7%, felt that their big data initiatives have been successful. Only 1.6% said their efforts were a failure; for some it’s still too early to tell.
This is great news. Big data programs aren’t cheap—37% reported that their organization has invested more than $100MM on big data initiatives, and 6.5% spent over $1B—so it’s wonderful that they are paying off. Big data has been fairly high on the hype-o-meter, so it’s also good to see that there is real substance to it. The companies in the survey are generally early adopters of new technologies (95% said they’d undertaken a big data initiative over the past five years), but it bodes well for smaller and more conservative firms that the early work on big data is going well. The respondents (mostly financial services giants) are heavy users of data and at the forefront of adoption when it comes to experimenting with data and analytics capabilities through centers of excellence, big data labs, analytics sandboxes, and the like.
This doesn’t mean, of course, that firms haven’t encountered some obstacles along the way. The primary impediment to big data progress seems not to be hardware or software, but “wetware”—human and organizational factors (Figure 1). 43% of surveyed companies mentioned “lack of organizational alignment” as an impediment. 41% pointed specifically to middle management as the culprit; the same percentage faulted “business resistance or lack of understanding.” 29.5% said their organizations lacked a coherent data strategy. 86% say their companies have tried to create a data-driven culture, but only 37% say they’ve been successful at it.
And some objectives have been more likely to be achieved than others (Figure 2).


