Clint Hook, director of Data Governance at Experian, looks at how organisations can automate data quality to support artificial intelligence and machine learning
Over the last decade, companies have begun to grasp and unlock the potential that artificial intelligence (AI) and machine learning (ML) can bring. While still in its infancy, companies are starting to understand the significant impact this technology can bring, helping them make better, faster and more efficient decisions.
Of course, AI and ML is no silver bullet to help businesses embrace innovation. In fact, the success of these algorithms is only as good as their foundations — specifically, quality data.
Without it, businesses will see the very objective they’ve installed AI and ML to do fail, with the unforeseen consequences of bad data causing irreversible damage to the business both in terms of its efficiency and reputation.
But there’s another area of exploration which is ripe for development; namely, can data quality be improved and maintained by automation and machine learning itself?
From movie streaming services, to chatbots, to helping inform how supermarkets arrange their shelves and guiding us through major transport hubs, ML influences our lives in ways that were unimaginable a decade ago.
But what happens if the algorithm is set to work on the foundation of poor data quality? The risks in the future could be far more severe than being served a film you don’t like.
If we begin trusting machine learning to improve the discovery and testing of pharmaceuticals, for example, what would happen if a drug were formulated but there were errors in the chemical compound data used to simulate testing? The implications could be grave.
An emerging application of ML which could also be impacted by poor base data is self-driving vehicles. From maps and addresses to how a vehicle reacts to a cyclist, the data used to teach the machine will be crucial to consumer and regulator adoption.
ML algorithms – those sets of rules and calculations that help solve defined problems — can either support the improvement of data quality or be thrown off by inaccurate data should the possibility of poor data not be considered in their construction.
As with any digital transformation, moving from manual to automated and then ‘intelligent’ data quality management will require a long-term plan. Experian has identified four stages about the progression of data management, which we call the Data Management Maturity Curve. Unaware, Reactive, Proactive and Optimised & Governed reflect the four stages that span a full cycle of a data quality strategy.
The assessment has revealed a steady progression up the maturity curve, as organisations begin to release the potential of the data they hold and take it more seriously.
