For many years the Dutch police has used a risk modeling algorithm to predict the chance that an individual suspect will commit a violent crime. Follow the Money exposed the total lack of a moral, legal, and statistical justification for its use, and now the police has stopped using the system.
This animation is an explanation of the system (made by the Dutch police):
Initially the police explicitly used ethnic background as one of the factors to calculate the risk. People with an Antillean, Moroccan, or Somali background got a higher score. The police stopped using that data as input in 2017 (apparently out of its own accord). But they haven’t monitored or evaluated the statistical quality of the results since then and so have no idea whether the system continues to have an ethnic bias. Unfortunately, this lackadaisical approach seems to be the modus operandi of the Dutch police.
Professor Marc Schuilenburg explains in the piece how we have become a “precautionary society”: It used to be that there was first suspicion and then surveillance, but nowadays we first do the surveillance and then create the suspicion. According to Schuilenburg, science knows that it is basically impossible to make individual crime predictions. He therefore argues that police has to stop using this system.
That is exactly what the police now has done after a media-triggered quick evaluation. After years of use, and wrongly criminalising an untold number of people, they have come to the conclusion that the system isn’t actually useful.
See: Dubieus algoritme van de politie ‘voorspelt’ wie in de toekomst geweld zal plegen and Politie stopt met gewraakt algoritme dat ‘voorspelt’ wie in de toekomst geweld gebruikt at Follow the Money.