Category: Facial Recognition and Biometrics

Late Night Talks: Studenten slepen universiteit voor de rechter vanwege discriminerende AI-software

Vrije Universiteit Amsterdam student Robin Pocornie en Naomi Appelman, co-founder van non-profitorganisatie Racism and Technology Center, gaan met elkaar in gesprek over discriminatie binnen kunstmatige intelligentie (artificial intelligence). Wat zijn de voor- en nadelen van kunstmatige intelligentie en in hoeverre hebben we grip en hoe kunnen we discriminatie tegengaan in de snelle ontwikkelingen van technologie?

By Charisa Chotoe, Naomi Appelman and Robin Pocornie for YouTube on December 3, 2023

Automating apartheid in the Occupied Palestinian Territories

Security cameras on a wall inside the Old City of Jerusalem.

In this interview, Matt Mahmoudi explains the Amnesty report titled Automating Apartheid, which he contributed to. The report exposes how the Israeli authorities extensively use surveillance tools, facial recognition technologies, and networks of CCTV cameras to support, intensify and entrench their continued domination and oppression of Palestinians in the Occupied Territories (OPT), Hebron and East Jerusalem. Facial recognition software is used by Israeli authorities to consolidate existing practices of discriminatory policing and segregation, violating Palestinians’ basic rights.

Continue reading “Automating apartheid in the Occupied Palestinian Territories” →

Judgement of the Dutch Institute for Human Rights shows how difficult it is to legally prove algorithmic discrimination

Robin Pocornie, a Black woman with a bald head and glasses, with fabulous nails, sitting behind her laptop which shows her face from the webcam.

On October 17th, the Netherlands Institute for Human Rights ruled that the VU did not discriminate against bioinformatics student Robin Pocornie on the basis of race by using anti-cheating software. However, according to the institute, the VU has discriminated on the grounds of race in how they handled her complaint.

Continue reading “Judgement of the Dutch Institute for Human Rights shows how difficult it is to legally prove algorithmic discrimination” →

Waarom we zwarte vrouwen meer zouden moeten geloven dan techbedrijven

Stel je voor dat bedrijven technologie bouwen die fundamenteel racistisch is: het is bekend dat die technologie voor zwarte mensen bijna 30 procent vaker niet werkt dan voor witte mensen. Stel je vervolgens voor dat deze technologie wordt ingezet op een cruciaal gebied van je leven: je werk, onderwijs, gezondheidszorg. En stel je tot slot voor dat je een zwarte vrouw bent en dat de technologie werkt zoals verwacht: niet voor jou. Je dient een klacht in. Om vervolgens van de nationale mensenrechteninstantie te horen dat het in dit geval waarschijnlijk geen racisme was.

By Nani Jansen Reventlow for Volkskrant on October 22, 2023

Judgement of the Dutch Institute for Human Rights shows how difficult it is to legally prove algorithmic discrimination

Robin Pocornie, a Black woman with a bald head and glasses, with fabulous nails, sitting behind her laptop which shows her face from the webcam.

Today, the Netherlands Institute for Human Rights ruled that the VU did not discriminate against bioinformatics student Robin Pocornie on the basis of race by using anti-cheating software. However, the VU has discriminated on the grounds of race when handling her complaint.

Continue reading “Judgement of the Dutch Institute for Human Rights shows how difficult it is to legally prove algorithmic discrimination” →

Uitspraak College voor de Rechten van de Mens laat zien hoe moeilijk het is om algoritmische discriminatie juridisch te bewijzen

Robin Pocornie, a Black woman with a bald head and glasses, with fabulous nails, sitting behind her laptop which shows her face from the webcam.

Vandaag heeft het College van de Rechten van de Mens geoordeeld dat de VU de student bioinformatica Robin Pocornie niet heeft gediscrimineerd op basis van ras door de inzet van antispieksoftware. Wel heeft de VU verboden onderscheid op grond van ras gemaakt bij de klachtbehandeling.

Continue reading “Uitspraak College voor de Rechten van de Mens laat zien hoe moeilijk het is om algoritmische discriminatie juridisch te bewijzen” →

Zwarte mensen vaker niet herkend door antispieksoftware Proctorio

Gezichten van mensen met een zwarte huidskleur worden veel minder goed herkend door tentamensoftware Proctorio, blijkt uit onderzoek van RTL Nieuws. De software, die fraude moet herkennen, zoekt bij online tentamens naar het gezicht van een student. Dat zwarte gezichten beduidend slechter worden herkend, leidt tot discriminatie, zeggen deskundigen die het onderzoek van RTL Nieuws beoordeelden.

By Stan Hulsen for RTL Nieuws on October 7, 2023

Proctoring software uses fudge-factor for dark skinned students to adjust their suspicion score

An image from the patent application that shows webcam shots of a test taker with different flags of suspicious behaviour.

Respondus, a vendor of online proctoring software, has been granted a patent for their “systems and methods for assessing data collected by automated proctoring.” The patent shows that their example method for calculating a risk score is adjusted on the basis of people’s skin colour.

Continue reading “Proctoring software uses fudge-factor for dark skinned students to adjust their suspicion score” →

Al Jazeera asks: Can AI eliminate human bias or does it perpetuate it?

Presenter Samantha Johnson starting the Digital Dilemma online show about ‘AI & Discrimination’

In its online series of digital dilemmas, Al Jazeera takes a look at AI in relation to social inequities. Loyal readers of this newsletter will recognise many of the examples they touch on, like how Stable Diffusion exacerbates and amplifies racial and gender disparities or the Dutch childcare benefits scandal.

Continue reading “Al Jazeera asks: Can AI eliminate human bias or does it perpetuate it?” →

Vooral vrouwen van kleur klagen de vooroordelen van AI aan

Wat je in zelflerende AI-systemen stopt, krijg je terug. Technologie, veelal ontwikkeld door witte mannen, versterkt en verbergt daardoor de vooroordelen. Met name vrouwen (van kleur) luiden de alarmbel.

By Marieke Rotman, Nani Jansen Reventlow, Oumaima Hajri and Tanya O’Carroll for De Groene Amsterdammer on July 12, 2023

France wants to legalise mass surveillance for the Paris Olympics 2024: “Safety” and “security”, for whom?

A wall of surveillance cameras all pointing in the same direction.

Many governments are using mass surveillance to support law enforcement for the purposes of safety and security. In France, the French Parliament (and before, the French Senate) have approved the use of automated behavioural video surveillance at the 2024 Paris Olympics. Simply put, France wants to legalise mass surveillance at the national level which can violate many rights, such as the freedom of assembly and association, privacy, and non-discrimination.

Continue reading “France wants to legalise mass surveillance for the Paris Olympics 2024: “Safety” and “security”, for whom?” →

Your Voice is (Not) Your Passport

In summer 2021, sound artist, engineer, musician, and educator Johann Diedrick convened a panel at the intersection of racial bias, listening, and AI technology at Pioneerworks in Brooklyn, NY. Diedrick.

By Michelle Pfeifer for Sounding Out! on June 12, 2023

Attempts to eliminate bias through diversifying datasets? A distraction from the root of the problem

Nine different images of Hito Steyerls as found via haveibeentrained.com

In this eloquent and haunting piece by Hito Steyerl, she weaves the ongoing narratives of the eugenicist history of statistics with its integration into machine learning. She elaborates why the attempts to eliminate bias in facial recognition technology through diversifying datasets obscures the root of the problem: machine learning and automation are fundamentally reliant on extracting and exploiting human labour.

Continue reading “Attempts to eliminate bias through diversifying datasets? A distraction from the root of the problem” →

Consensus and subjectivity of skin tone annotation for ML fairness

Skin tone is an observable characteristic that is subjective, perceived differently by individuals (e.g., depending on their location or culture) and thus is complicated to annotate. That said, the ability to reliably and accurately annotate skin tone is highly important in computer vision. This became apparent in 2018, when the Gender Shades study highlighted that computer vision systems struggled to detect people with darker skin tones, and performed particularly poorly for women with darker skin tones. The study highlights the importance for computer researchers and practitioners to evaluate their technologies across the full range of skin tones and at intersections of identities. Beyond evaluating model performance on skin tone, skin tone annotations enable researchers to measure diversity and representation in image retrieval systems, dataset collection, and image generation. For all of these applications, a collection of meaningful and inclusive skin tone annotations is key.

By Candice Schumann and Gbolahan O. Olanubi for Google AI Blog on May 15, 2023

Mean Images

An artist considers a new form of machinic representation: the statistical rendering of large datasets, indexed to the probable rather than the real of photography; to the uncanny composite rather than the abstraction of the graph.

By Hito Steyerl for New Left Review on April 28, 2023

Watching the watchers: bias and vulnerability in remote proctoring software

Educators are rapidly switching to remote proctoring and examination software for their testing needs, both due to the COVID-19 pandemic and the expanding virtualization of the education sector. State boards are increasingly utilizing these software for high stakes legal and medical licensing exams. Three key concerns arise with the use of these complex software: exam integrity, exam procedural fairness, and exam-taker security and privacy. We conduct the first technical analysis of each of these concerns through a case study of four primary proctoring suites used in U.S. law school and state attorney licensing exams. We reverse engineer these proctoring suites and find that despite promises of high-security, all their anti-cheating measures can be trivially bypassed and can pose significant user security risks. We evaluate current facial recognition classifiers alongside the classifier used by Examplify, the legal exam proctoring suite with the largest market share, to ascertain their accuracy and determine whether faces with certain skin tones are more readily flagged for cheating. Finally, we offer recommendations to improve the integrity and fairness of the remotely proctored exam experience.

By Avi Ginsberg, Ben Burgess, Edward W. Felten and Shaanan Cohney for arXiv.org on May 6, 2022

Enough is Enough. Tell Congress to Ban Federal Use of Face Recognition

Cities and counties across the country have banned government use of face surveillance technology, and many more are weighing proposals to do so. From Boston to San Francisco, Jackson, Mississippi to Minneapolis, elected officials and activists know that face surveillance gives police the power to track us wherever we go. It also disproportionately impacts people of color, turns us all into perpetual suspects, increases the likelihood of being falsely arrested, and chills people’s willingness to participate in first amendment protected activities. Even Amazon, known for operating one of the largest video surveillance networks in the history of the world, extended its moratorium on selling face recognition to police.

By Matthew Guariglia for Electronic Frontier Foundation (EFF) on April 4, 2023

Ben jij straks werkloos door AI?

Het einde van 2022 stond in het teken van de AI-tools. Je maakt digitale kunstwerken met DALL-E, AI-profielfoto’s met Lensa en als klap op de vuurpijl genereer je binnen een paar seconden een hele sollicitatiebrief of essay via ChatGPT. Dat AI, of kunstmatige intelligentie, veel kan wisten we. Maar ChatGPT wordt echt gezien als een doorbraak. Wat is het? En worden wij overbodig door AI? Oh en Devran dacht trouwens lekker ontspannen het nieuwe jaar in te gaan met de chatbot, maar of dat nou zo’n goed idee was…

By Robin Pocornie for YouTube on December 31, 2022

Dutch Institute for Human Rights: Use of anti-cheating software can be algorithmic discrimination (i.e. racist)

Dutch student Robin Pocornie filed a complaint with Dutch Institute for Human Rights. The surveillance software that her university used, had trouble recognising her as human being because of her skin colour. After a hearing, the Institute has now ruled that Robin has presented enough evidence to assume that she was indeed discriminated against. The ball is now in the court of the VU (her university) to prove that the software treated everybody the same.

Continue reading “Dutch Institute for Human Rights: Use of anti-cheating software can be algorithmic discrimination (i.e. racist)” →

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑