Last year, Investico revealed how DUO, the Dutch organization for administering student grants, was using a racist algorithm to decide which students would get a home visit to check for fraudulent behaviour. The Minister of Education immediately stopped the use of the algorithm.
Further research concluded that indirect discrimination had taken place, forcing the Dutch government into an apology and a promise of further research into the scale of the problem. That research, by Algorithm Audit, has now been published.
The findings are shocking (translated from the Dutch):
- The process has been biased overall against students with a non-European migration background.
- The risk profile that was used in the process was biased against students with a non-European migration background. The cause of this was assigning a higher risk to vocational students and students who were registered close to their parents’ address.
- Manual selection reinforced the bias of the process.
- Because the CUB process was biased against students with a non-European migration background, a lot of improper use was identified in this group. This is largely the result of excessive scrutiny of this demographic.
- The group of students who object to the determination of improper use consists overwhelmingly of students with a non-European migration background. No bias has been identified in the objection procedure itself.
All of this means that “students with a non-European migration background had a 3.0 times higher chance of receiving a home visit from which no improper use was later found compared to students of Dutch origin,” and that “the population filing objections consists of 79-85% students with a non-European migration background.”
You can read the full Algorithm Audit report here.
This begs the question: now what? The Minister has organised conversations with students about the impact that checks for fraud have had on them, and DUO is trying to come to a more equitable process for checking students. But no words so far about all the students that have been accused wrongly of fraud and didn’t have the wherewithal to object. More news in the fall.
See: Vervolgonderzoek bevestigt indirecte discriminatie controles uitwonendenbeurs at DUO.
Image from the cover of the Algorithm Audit report.