As businesses in virtually every industry move to algorithmic decision making, the subject of fairness in these decisions are often completely ignored or left as an afterthought. In the emerging data economy where innovation and competitive strategy will be driven by the “data advantage” and what companies can do with data, it would be irresponsible not to consider the implications that these algorithms can have on individuals and society.
The more data you feed to pattern recognition algorithms the better it is able to discriminate between groups [1] and are rewarded for doing so. Although these so called accurate predictive algorithms could create short term value, the Bias produced in their results could potentially cause unwanted harm to some groups and prove to be a huge risk to organizations.
However, companies can adopt key strategies from the onset to combat these problems to realize the true potential that lies in AI and automated decision making.
Al can consume huge amounts of data, make faster and more consistent decisions than humans, leading to massive gains in productivity and efficiency. However, the “move fast and break things” philosophy adopted by technology companies in the last decade has proved to be ill-fitting to AI applications, as is extensively documented by data journalists and independent research institutions. ProPublica, a nonprofit organization working in investigative journalism, showed in the Pulitzer prize finalist series “Machine Bias” how an algorithm designed to predict future criminals is biased against certain ethnic groups [2] and in another, car insurance companies are shown to charge more premium to minority neighbourhoods than white areas with the same risk [3]. These results may not always be the result of disparage treatment, for AI algorithms that are optimized to identify high value customer would accordingly discriminate against low income groups. Removing dimensions that identify historically discriminated groups such as gender, race or disability does not always work since the existing bias in the data can still help find good proxies to identify these groups.
The pervasive use of surveillance cameras and facial recognition technology is also concerning. In the city of Zhengzhou, Chinese authorities are using facial recognition to keep tabs on the Uighurs, a Muslim minority group. Developments in affect recognition, subclass of facial recognition that claims to detect things such as personality, inner feelings, mental health, and “worker engagement” based on images or video of faces, when linked to hiring or policy decisions threatens to bring back physiognomic ideas from the Nazi era. [4]
Understanding the effects of these systems can sound complex and esoteric but systematic efforts by experts, who understand AI and regulations, and thorough analysis of each part of the Data Science process can help provide a clearer picture of the expected outcomes before organizations can start deploying these models.
In April 2016, the European Parliament adopted a set of comprehensive regulations for the collection, storage and use of personal information, the General Data Protection Regulation (GDPR) [5], which would go into effect from May 2018.
Article 22 (Automated individual decision making, including profiling) and Article 13-15 are of particular interest to organizations looking to venture into AI or that have already deployed AI models as these provisions could potentially cause an overhaul of popular techniques used in recommendation systems, credit and insurance risk assessments,
Article 22 specifically addresses discrimination from profiling that makes use of sensitive data. As previously discussed, removal of these sensitive information that code for race, finances, or any of the other categories of sensitive information referred to in Article 9 does not ensure compliance since other correlated variables can form proxies for these “special categories of personal data”.
