The future will be data-driven

Infrastructure serves a purpose: transport systems provide mobility and accessibility; utilities provide energy and water. None of this is possible without the surrounding infrastructure. But now there is a new utility, so crucial to the efficient functioning of modern life that we could not do without it: data. So much of how we live, and how the economy functions, is driven by data. And just like traditional utilities, it needs the proper infrastructure: to collect it, store it and move it around to the points of use. This needs smart administration, sensors, computer storage, fibre and wireless communication networks, analytics and display. The key point, as with other infrastructure, is to work backwards along this sequence: to ask “how do we want to use data, and what infrastructure will allow us to do that most efficiently?”

It was Claude Shannon in his famous 1948 paper on “A Mathematical Theory of Communication” who first established how to encode information in “bits,” work that provided the conceptual basis for the transmission of data. The ensuing technological advances throughout the 20th century have influenced business and government. But data and processor power have never been as prevalent and powerful as they are today. It is the internet that has transformed how businesses and services work, and how we live: administration, manufacturing, investment and planning, sales and delivery have all changed beyond recognition.

But there is more to come—the fields of data science and Artificial Intelligence will be transformative in new ways. What has been achieved so far is largely in specialised silos. The challenge is to explore the future of the big systems, for example in science, engineering, health, education and security; and how these combine in the places where most of us live: in cities or city regions.

If we take the growth of computing power and the continued expansion of communications bandwidth as given, there are two parts of data science that underpin future developments in applied domains: Data wrangling and data analytics.

Data systems are messy. Take one example: medical data. This can range from the simple, say, a handwritten doctor’s note, through to the highly-complex results of an MRI scan. Data wrangling is the task of digitising all of this and making it available in common formats. In this medical example, this organised data system can be analysed using machine learning algorithms—a series of computerised instructions—and social data on the patient to provide an entirely new level of medical insight into a patient’s condition. Indeed, there are already examples where the algorithm may provide a quicker or better diagnosis than a doctor.

Messy data makes data wrangling a very time consuming but necessary process. It is estimated that this can take 80 per cent of the time in developing data analysis applications. A key research challenge, therefore, is to find a way of automating this.

And once the data has been gathered together—what then? The modern data analytics tool kit contains a broad array of useful instruments. In some cases, this may be a matter of traditional statistical analysis. In other cases, mathematical models can be used to explore different hypothetical scenarios. This is commonplace in, for example, transport planning and retail store location. The ability to test out scenarios in advance saves a huge amount of potentially wasted investment capital, thus freeing it up for use elsewhere.

These capabilities—of data wrangling and analysis—have been combined to produce what are termed machine learning (ML) algorithms. These can be applied to data sets to “learn” routine business processes. So perhaps they might recommend whether to accept a person for insurance and if so, what premium that they should pay each month, and all this without human intervention.