Trust in data won’t come until people take responsibility for it

Big data has long been heralded as the ultimate 21st-century game changer. But as the development of data science rushes ahead, we are increasingly expected to put our faith into things we can’t control, writes Elaine Burke.

Since the term ‘big data’ was coined, we’ve been hearing from technology evangelists how great masses of data will be used to vastly improve how we run the world. Now that tools such as algorithms, artificial intelligence (AI) and machine learning are more and more readily at our disposal, that potential is beginning to be realised. Unfortunately, however, some of the stories emerging about how data is being used don’t yet paint a hopeful picture.

Right now, Netflix documentary The Social Dilemma is racking up viewers interested to learn how social media makes use of data, with insights from the very people who built these platforms. The film’s dramatisations seek to demonstrate how the data we feed into social media is used to trigger addictive behaviour, casting Mad Men’s Vincent Kartheiser as a multifaceted algorithm programmed to continually optimise engagement and advertising.

The power of persuasion is nothing new in the advertising world but the issue raised by The Social Dilemma and other critiques of the online industry is that advertising in other media can be strictly regulated, but you can’t feasibly control advertising at the scale and level of personalisation stemming from online engines.

Social media has borne the brunt of most coverage of data abuses, but data is being used in problematic ways elsewhere. There are regular reports of biased algorithms let loose in the wild to make flawed decisions, particularly against people who don’t fall into the rich, white male demographic.

Earlier this year, Abeba Birhane of Science Foundation Ireland’s software research centre Lero helped uncover how MIT’s much-cited ‘80 Million Tiny Images’ dataset may have contaminated AI systems with racist and misogynistic terms.

This dataset has been available to train machine learning systems since 2008, so that’s more than a decade of problematic data being used blindly by the community responsible for advancing this decision-making technology. Because who has time to vet the quality of tens of millions of data entries?

In response to this report, MIT has retired the dataset and discouraged its use. Birhane, meanwhile, has called for “radical ethics” and the careful consideration of direct and indirect impacts of using such datasets in future, particularly for vulnerable groups.

The risk of social bias being embedded in data came to the fore in debates over how to allocate grades to students who could not sit traditional exams during the Covid-19 pandemic. In the UK, a grading system built to make use of the data available on students and their schools was widely criticised for how it could impact students from disadvantaged communities and, overall, the roll-out of predicated grades determined by an algorithm was a disaster for the UK government.

The Irish Government tried to sidestep this pitfall by relying only on teacher assessments and students’ previous performance in State examinations for its calculated grades. However, there wasn’t enough time to code and sufficiently test such a system and the result is that grave errors were discovered after the grades were already published.

In this case, the Government’s decision to have the calculated grading code developed in secrecy only exasperated its issues with time and testing. Had it taken a transparent and open-source approach to development, it would have benefitted from the help of many experienced hands on a project of significant public importance.