Category: Algorithmic Bias

Long overdue: Google has improved its camera app to work better for Black people

The following short video by Vox shows how white skin has always been the norm in photography. Black people didn’t start to look good on film until in the 1970s furniture makers complained to Kodak that their film didn’t render the difference between dark and light grained wood, and chocolate companies were upset that you couldn’t see the difference between dark and light chocolate.

Continue reading “Long overdue: Google has improved its camera app to work better for Black people” →

Sentenced by Algorithm

Computer programs used to predict recidivism and determine prison terms have a high error rate, a secret design, and a demonstrable racial bias.

By Jed S. Rakoff for The New York Review of Books on June 10, 2021

Racist and classist predictive policing exists in Europe too

The enduring idea that technology will be able to solve many of the existing problems in society continues to permeate across governments. For the EUObserver, Fieke Jansen and Sarah Chander illustrate some of the problematic and harmful uses of ‘predictive’ algorithmic systems by states and public authorities across the UK and Europe.

Continue reading “Racist and classist predictive policing exists in Europe too” →

Algorithmic discrimination in Europe : challenges and opportunities for gender equality and non-discrimination law.

This report investigates how algorithmic discrimination challenges the set of legal guarantees put in place in Europe to combat discrimination and ensure equal treatment. More specifically, it examines whether and how the current gender equality and non-discrimination legislative framework in place in the EU can adequately capture and redress algorithmic discrimination. It explores the gaps and weaknesses that emerge at both the EU and national levels from the interaction between, on the one hand, the specific types of discrimination that arise when algorithms are used in decision-making systems and, on the other, the particular material and personal scope of the existing legislative framework. This report also maps out the existing legal solutions, accompanying policy measures and good practice to address and redress algorithmic discrimination both at EU and national levels. Moreover, this report proposes its own integrated set of legal, knowledge-based and technological solutions to the problem of algorithmic discrimination.

By Janneke Gerards and Raphaële Xenidis for Publication Office of the European Union on March 10, 2021

Rotterdam’s use of algorithms could lead to ethnic profiling

The Rekenkamer Rotterdam (a Court of Audit) looked at how the city of Rotterdam is using predictive algorithms and whether that use could lead to ethical problems. In their report, they describe how the city lacks a proper overview of the algorithms that it is using, how there is no coordination and thus no one takes responsibility when things go wrong, and how sensitive data (like nationality) were not used by one particular fraud detection algorithm, but that so-called proxy variables for ethnicity – like low literacy, which might correlate with ethnicity – were still part of the calculations. According to the Rekenkamer this could lead to unfair treatment, or as we would call it: ethnic profiling.

Continue reading “Rotterdam’s use of algorithms could lead to ethnic profiling” →

Gebruik algoritmes Rotterdam kan leiden tot vooringenomen uitkomsten

De gemeente Rotterdam maakt ter ondersteuning van haar besluitvorming gebruik van algoritmes. Hoewel er binnen de gemeente aandacht bestaat voor het ethisch gebruik van algoritmes, is het besef van de noodzaak hiervan nog niet heel wijdverbreid. Dit kan leiden tot weinig transparantie van algoritmes en vooringenomen uitkomsten, zoals bij een algoritme gericht op de bestrijding van uitkeringsfraude. Dit en meer concludeert de Rekenkamer Rotterdam in het rapport ‘Gekleurde technologie’.

From Rekenkamer Rotterdam on April 14, 2021

Online proctoring excludes and discriminates

The use of software to automatically detect cheating on online exams – online proctoring – has been the go-to solution for many schools and universities in response to the COVID-19 pandemic. In this article, Shea Swauger addresses some of the potential discriminatory, privacy and security harms that can impact groups of students across class, gender, race, and disability lines. Swauger provides a critique on how technologies encode “normal” bodies – cisgender, white, able-bodied, neurotypical, male – as the standard and how students who do not (or cannot) conform, are punished by it.

Continue reading “Online proctoring excludes and discriminates” →

The Dutch government’s love affair with ethnic profiling

In his article for One World, Florentijn van Rootselaar shows how the Dutch government uses automated systems to profile certain groups based on their ethnicity. He uses several examples to expose how, even though Western countries are often quick to denounce China’s use of technology to surveil, profile and oppress the Uighurs, the same states themselves use or contribute to the development of similar technologies.

Continue reading “The Dutch government’s love affair with ethnic profiling” →

Racist technology in action: Gun, or electronic device?

The answer to that question depends on your skin colour, apparently. An AlgorithmWatch reporter, Nicholas Kayser-Bril, conducted an experiment that went viral on Twitter, showing that Google Vision Cloud (a service which is based on a subset of AI known as “computer vision” that focuses on automated image labelling), labelled an image of a dark-skinned individual holding a thermometer with the word “gun”, whilst a lighter skinned individual was labelled holding an “electronic device”.

Continue reading “Racist technology in action: Gun, or electronic device?” →

Decode the Default

Technology has never been colorblind. It’s time to abolish notions of “universal” users of software.

From The Internet Health Report 2020 on January 1, 2021

Google fires AI researcher Timnit Gebru

Google has fired AI researcher and ethicist Timnit Gebru after she wrote an email criticising Google’s policies around diversity while she struggled with her leadership to get a critical paper on AI published. This angered thousands of her former colleagues and academics. They pointed at the unequal treatment that Gebru received as a black woman and they were worried about the integrity of Google’s research.

Continue reading “Google fires AI researcher Timnit Gebru” →

Hoe Nederland A.I. inzet voor etnisch profileren

China dat kunstmatige intelligentie inzet om Oeigoeren te onderdrukken: klinkt als een ver-van-je-bed-show? Ook Nederland (ver)volgt specifieke bevolkingsgroepen met algoritmes. Zoals in Roermond, waar camera’s alarm slaan bij auto’s met een Oost-Europees nummerbord.

By Florentijn van Rootselaar for OneWorld on January 14, 2021

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑