For minorities, biased AI algorithms can damage almost every part of life

In his new book, ‘Is Artificial Intelligence Racist?’, Arshin Adib-Moghaddam explores the relationship between various forms of racism and sexism and artificial intelligence.

AI ethics

Arshin Adib-Moghaddam


September 6, 2023

Bad data does not only produce bad outcomes. It can also help to suppress sections of society, for instance vulnerable women and minorities.

This is the argument of my new book on the relationship between various forms of racism and sexism and artificial intelligence (AI). The problem is acute. Algorithms generally need to be exposed to data – often taken from the internet – in order to improve at whatever they do, such as screening job applications, or underwriting mortgages.

But the training data often contains many of the biases that exist in the real world. For example, algorithms could learn that most people in a particular job role are male and therefore favour men in job applications. Our data is polluted by a set of myths from the age of “enlightenment”, including biases that lead to discrimination based on gender and sexual identity.

Judging from the history in societies where racism has played a role in establishing the social and political order, extending privileges to white males –- in Europe, North America and Australia, for instance –- it is simple science to assume that residues of racist discrimination feed into our technology.

In my research for the book, I have documented some prominent examples. Face recognition software more commonly misidentified black and Asian minorities, leading to false arrests in the US and elsewhere.

Software used in the criminal justice system has predicted that black offenders would have higher recidivism rates than they did. There have been false healthcare decisions. A study found that of the black and white patients assigned the same health risk score by an algorithm used in US health management, the black patients were often sicker than their white counterparts.

This reduced the number of black patients identified for extra care by more than half. Because less money was spent on black patients who have the same level of need as white ones, the algorithm falsely concluded that black patients were healthier than equally sick white patients. Denial of mortgages for minority populations is facilitated by biased data sets. The list goes on.

Machines don’t lie?

Such oppressive algorithms intrude on almost every area of our lives. AI is making matters worse, as it is sold to us as essentially unbiased. We are told that machines don’t lie. Therefore, the logic goes, no one is to blame.

This pseudo-objectiveness is central to the AI-hype created by the Silicon Valley tech giants. It is easily discernible from the speeches of Elon Musk, Mark Zuckerberg and Bill Gates, even if now and then they warn us about the projects that they themselves are responsible for.

There are various unaddressed legal and ethical issues at stake. Who is accountable for the mistakes? Could someone claim compensation for an algorithm denying them parole based on their ethnic background in the same way that one might for a toaster that exploded in a kitchen?

The opaque nature of AI technology poses serious challenges to legal systems which have been built around individual or human accountability. On a more fundamental level, basic human rights are threatened, as legal accountability is blurred by the maze of technology placed between perpetrators and the various forms of discrimination that can be conveniently blamed on the machine.

Racism has always been a systematic strategy to order society. It builds, legitimises and enforces hierarchies between the haves and have nots.