Turning big data into business insights: The state of play

We live in an increasingly data-driven society, in which information is becoming as much of a currency as money. Many consumers use free services from internet giants like Google, Facebook, Amazon, Microsoft and Apple, for example, and in return allow these corporations to track and monetise their online behaviour.

One of the biggest questions of the day is the openness of such transactions, and the level of control that individuals have over the fate of the personal information they — sometimes unwittingly — divulge to organisations with which they interact online. Recent votes on both sides of the Atlantic have highlighted the capacity for data-savvy organisations to hoover up and profile large amounts of user data — including demographics, consumer behaviour and internet activity — in order to micro-target adverts, news stories and services in support of particular goals or causes.

Clearly, the data floodgates are now opening for businesses of all sizes and descriptions, bringing myriad opportunities for timely analysis in pursuit of competitive advantage. Although the focus is currently slanted towards customer behaviour, data is available at multiple points in the product or service supply chain, and comes in many forms — traditional (structured), ad hoc (unstructured), real time, and IoT- or M2M-generated, to name but a few.

Companies that implement big data analytics successfully can reap rich rewards from cost-saving efficiencies and revenue-generating innovations. This can help businesses achieve a digital transformation, allowing them to maintain competitiveness in the face of any disruptive startups — which are data-driven almost by definition — that spring up in their markets.

However, useful business insights don’t automatically flow from a torrent of heterogeneous information: actionable data must be identified, organised and analysed, and the results implemented across relevant parts of the business. That requires planning, budget and the right tools and expertise.

This overview, and the remainder of this ZDNet special report, examines the state of play in big data analytics. We may have passed ‘peak hype’ on the subject — analyst firm Gartner dropped Big Data from its Hype Cycle for Emerging Technologies back in 2015 — but has it yet delivered on its promise?

Attempts are periodically made to estimate how much data is generated worldwide every year, and in what form. Back in 2014, IDC and EMC put the ‘Digital Universe’ at 4.4 zettabytes (ZB) in 2013 — that’s 4.4 trillion gigabytes — and predicted this would grow to 44ZB in 2020, more than doubling every two years. The latest estimate, from IDC and Seagate’s Data Age 2025 report, puts the 2025 figure (now dubbed the ‘Global Datasphere’) at 163ZB — a tenfold rise from the 16.1ZB created in 2016.

The IDC/Seagate report also predicts that the bulk of worldwide data creation will shift from consumers to enterprises, the latter accounting for 60 percent by 2025. Trends driving this shift, according to the report, include: the evolution of data from business background to life-critical; embedded systems and the IoT; cognitive/AI systems that change the landscape; mobile and real-time data; and security as a critical foundation.

All that data needs a home, either permanent or temporary, which explains the interest of a storage company like Seagate in this area.

In a statement launching the report, Seagate CEO Steve Luczo (soon to become Executive Chairman) said: “While we can see from this new research that the era of Big Data is upon us, the value of data is really not in the ‘known’, but in the ‘unknown’ where we are vastly underestimating the potentials today. What is really exciting are the analytics, the new businesses, the new thinking and new ecosystems from industries like robotics and machine-to-machine learning, and their profound social and economic impact on our society.