Dutch responses to the DUO scandal miss the point: technical fixes won’t solve the fundamental problems of predictive optimisation

The DUO discrimination scandal, where more than 10.000 students were discriminated against, has led to multiple initiatives that aim to prevent this from happening again. None of these addresses the core problems of “predictive optimisation”.

Continue reading “Dutch responses to the DUO scandal miss the point: technical fixes won’t solve the fundamental problems of predictive optimisation”

Racist Technology in Action: Dutch banks racially profile Muslims and Palestinian activists, alleging “terrorist activity”

In another example of racial profiling in the Netherlands, banks such as ING, Rabobank, and ABN Amro have subjected Muslims and Palestinian activists to intrusive, discriminatory checks based on allegations of “terrorist financing.”

Continue reading “Racist Technology in Action: Dutch banks racially profile Muslims and Palestinian activists, alleging “terrorist activity””

50% of the profiling algorithms used by the Dutch tax office are discriminatory and therefore unlawful according to the Data Protection Authority

The Dutch tax office is plagued by one problem after another. The Child Benefits Scandal was supposed to be a wake-up call, but apparently, their system use is so atrophied that they can’t seem to do what is needed. Follow the Money reports on a letter from the Dutch data protection authority to the Minister of Finance, which argues that the tax office uses 50 algorithms that use discriminatory profiling and are therefore potentially unlawful.

Continue reading “50% of the profiling algorithms used by the Dutch tax office are discriminatory and therefore unlawful according to the Data Protection Authority”

New York City uses a secret Child Welfare Algorithm

New York City’s Administration for Children’s Services (ACS) has been secretly using an AI risk assessment system since 2018 to flag families for additional investigation. This Markup investigation reveals how this algorithm mainly affects families of colour and raises serious questions about algorithmic bias against racialised and poor families in child welfare.

Continue reading “New York City uses a secret Child Welfare Algorithm”

Racist Technology in Action: How the municipality of Amsterdam tried to roll out a ‘fair’ fraud detection algorithm. Spoiler alert: it was a disaster

Amsterdam officials’ technosolutionist way of thinking struck once again: they believed they could build technology that would prevent fraud while protecting citizens’ rights through their “Smart Check” AI system.

Continue reading “Racist Technology in Action: How the municipality of Amsterdam tried to roll out a ‘fair’ fraud detection algorithm. Spoiler alert: it was a disaster”

Dutch Institute for Human Rights creates an evaluation framework for risk profiling and urges organisations to do more to prevent discrimination based on race and nationality

The Dutch Institute for Human Rights has published an evaluation framework for risk profiling intending to prevent discrimination based on race or nationality.

Continue reading “Dutch Institute for Human Rights creates an evaluation framework for risk profiling and urges organisations to do more to prevent discrimination based on race and nationality”

In Spain, an algorithm used by police to ‘combat’ gender violence determines whether women live or die

Lobna Hemid. Stefany González Escarraman. Eva Jaular (and her 11-month-old baby). The lives of these three women and an infant, amongst many others, tragically ended due to gender-related killings in Spain. As reported in this article, they were all classified as “low” or “negligible” risk by VioGén, despite reporting abuse to the police. In the case of Lobna Hemid, after reporting her husband’s abuse to the police and being assessed as “low risk” by VioGén, the police provided her with minimal protection, and weeks later, her husband stabbed her to death.

Continue reading “In Spain, an algorithm used by police to ‘combat’ gender violence determines whether women live or die”

Students with a non-European migration background had a 3.0 times higher chance of receiving an unfounded home visit from the Dutch student grants fraud department

Last year, Investico revealed how DUO, the Dutch organization for administering student grants, was using a racist algorithm to decide which students would get a home visit to check for fraudulent behaviour. The Minister of Education immediately stopped the use of the algorithm.

Continue reading “Students with a non-European migration background had a 3.0 times higher chance of receiving an unfounded home visit from the Dutch student grants fraud department”

Dutch Ministry of Foreign Affairs dislikes the conclusions of a solid report that marks their visa process as discriminatory so buys a shoddy report saying the opposite

For more than a year now, the Dutch Ministry of Foreign Affairs has ignored advice from its experts and continued its use of discriminatory risk profiling of visa applicants.

Continue reading “Dutch Ministry of Foreign Affairs dislikes the conclusions of a solid report that marks their visa process as discriminatory so buys a shoddy report saying the opposite”

Dutch police used algorithm to predict violent behaviour without any safeguards

For many years the Dutch police has used a risk modeling algorithm to predict the chance that an individual suspect will commit a violent crime. Follow the Money exposed the total lack of a moral, legal, and statistical justification for its use, and now the police has stopped using the system.

Continue reading “Dutch police used algorithm to predict violent behaviour without any safeguards”

Racist Technology in Action: You look similar to someone we didn’t like → Dutch visa denied

Ignoring earlier Dutch failures in automated decision making, and ignoring advice from its own experts, the Dutch ministry of Foreign Affairs has decided to cut costs and cut corners through implementing a discriminatory profiling system to process visa applications.

Continue reading “Racist Technology in Action: You look similar to someone we didn’t like → Dutch visa denied”

The devastating consequences of risk based profiling by the Dutch police

Diana Sardjoe writes for Fair Trials about how her sons were profiled by the Amsterdam police on the basis of risk models (a form of predictive policing) called ‘Top600’ (for adults) and ‘Top400’ for people aged 12 to 23). Because of this profiling her sons were “continually monitored and harassed by police.”

Continue reading “The devastating consequences of risk based profiling by the Dutch police”

Racist Technology in Action: U.S. universities using race in their risk algorithms as a predictor for student success

An investigation by The Markup in March 2021, revealed that some universities in the U.S. are using a software and risk algorithm that uses the race of student as one of the factors to predict and evaluate how successful a student may be. Several universities have described race as a “high impact predictor”. The investigation found large disparities in how the software treated students of different races, with Black students deemed a four times higher risk than their White peers.

Continue reading “Racist Technology in Action: U.S. universities using race in their risk algorithms as a predictor for student success”

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑