In its report of the 25 of October, Amnesty slams the Dutch government’s use of discriminatory algorithms in the child benefits schandal (toeslagenaffaire) and warns that the likelihood of such a scandal occurring again is very high. The report is aptly titled ‘Xenophobic machines – Discrimination through unregulated use of algorithms in the Dutch childcare benefits scandal’ and it conducts a human rights analysis of a specific sub-element of the scandal: the use of algorithms and risk models. The report is based on the report of the Dutch data protection authority and several other government reports.
Amnesty’s main conclusion is that the ‘risk classification model’ used by the tax agency to score child benefits applicants on the likelihood of fraud, relied on racial profiling as nationality was a risk factor in this model. Amnesty additionally emphasizes the intersectional nature of this harm:
This treatment of parents and caregivers with non-Dutch nationalities by the Dutch tax authorities resulted in racial profiling. Different layers of inequality were created through discrimination based on people’s race and ethnicity in the use of the risk classification model, combined with the policies and practices that more often affected people with a lower economic status, who frequently belonged to an ethnic minority.
Based on its human rights assessment of the risk models used in the child benefits scandal, Amnesty calls for the creation of a substantive, supervisory and enforcement framework to prevent human rights violations through the government use of algorithmic or automated decision-making. Further, it calls or the prohibition of self learning algorithms such as the ones used in the scandal and for increased transparency. This last point is especially important in light of the new EU ‘AI Act’ that could potentially include a ban on the use of such technologies, as Nadia Benaissa from Bits of Freedom writes.
Even though this in depth analysis of the role of technology and algorithms in the scandal is an important contribution, it is important not to overstate this role. In the end government failure and abuse such as with the child benefits scandal is a product many different factors including systemic racism, personal failures of both civil servants and politicians, and a severe lack of accountability.
You can download the full English PDF of the report here.
See: Toeslagenaffaire is mensenrechtenschending, zegt Amnesty International