An investigation by The Markup in March 2021, revealed that some universities in the U.S. are using a software and risk algorithm that uses the race of student as one of the factors to predict and evaluate how successful a student may be. Several universities have described race as a “high impact predictor”. The investigation found large disparities in how the software treated students of different races, with Black students deemed a four times higher risk than their White peers.
The Markup found that at the University of Massachusetts Amherst, “Black women are 2.8 times as likely to be labeled high risk as White women, and Black men are 3.9 times as likely to be labeled high risk as White men.” At the University of Wisconsin–Milwaukee, “Black women [are of] high[er] risk at 2.2 times the rate of White women, and Black men at 2.9 times the rate of White men,” and at Texas A&M University, they labelled “Black women high risk at 2.4 times the rate of White women, and Black men at 2.3 times the rate of White men.”
The effect of these practices could mean that Black and other minority students will be pushed into “easier” majors by systems that are neither transparent nor reliable. The risk algorithms include other information such as test scores, high school percentiles, estimated skills, to name a few. These models are trained on historic student data, between two to ten or more years of student outcomes. Basing decisions on historic data, however, can encode and perpetuate un-investigated racist and discriminatory practices. It is also unclear how heavily race is weighted in the algorithms, as the universities have no insight into the proprietary software that they use.
A fundamental issue with using someone’s racial background as an indicator of risk is not only that it is stigmatising, but also that it is redundant. Race as a category cannot be changed, unlike an issue such as financial struggle. Hannah Quay-de la Vallee, a senior technologist at the Center for Democracy and Technology notes:
Using race for any kind of system, even if it is in a very narrow context of trying to end racial disparities in higher education … you can go into that with the best of intentions and then it takes very, very few steps to get you in place where you’re doing further harm.