The US Penal System

Comments like this remind us that people who are most directly impacted by the New Jim Code have a keen sense of the default discrimination facilitated by these technologies. As a form of social technology, institutional racism, past and present, is the precondition for the carceral technologies that underpin the US penal system. At every stage of the process – from policing, sentencing, and imprisonment to parole – automated risk assessments are employed to determine people’s likelihood of committing a crime.9 They determine the risk profile of neighborhoods in order to concentrate police surveillance, or the risk profile of individuals in order to determine whether or for how long to release people on parole.

In a recent study of the recidivism risk scores assigned to thousands of people arrested in Broward County, Florida, ProPublica investigators found that the score was remarkably unreliable in forecasting violent crime. They also uncovered significant racial disparities:

In forecasting who would re-offend, the algorithm made mistakes with black and white defendants at roughly the same rate but in very different ways. The formula was particularly likely to falsely

flag black defendants as future criminals, wrongly labeling them this way at almost twice the rate as white defendants. White defendants were mislabeled as low risk more often than black defendants.10

The algorithm generating the risk score builds upon already existing forms of racial domination and reinforces them precisely because the apparatus ignores how race shapes the “weather.” Literary scholar Christina Sharpe describes the weather as “the total climate; and the climate is antiblack.”11 For example, the survey given to prospective parolees to forecast the likelihood that they will recidivate includes questions about their criminal history, education and employment history, financial history, and neighborhood characteristics (among many other factors). As all these variables are structured by racial domination – from job market discrimination to ghettoization – the survey measures the extent to which an individual’s life chances have been impacted by racism without ever asking an individual’s race.12

Likewise, predictive policing software will always be more likely to direct police to neighborhoods like the one I grew up in, because the data that this software is drawing from reflect ongoing surveillance priorities that target predominantly Black neighborhoods.13 Anti- Blackness is no glitch. The system is accurately rigged, we might say, because, unlike in natural weather forecasts, the weathermen are also the ones who make it rain.14

Even those who purportedly seek “fairness” in algorithmic decision- making are not usually willing to assert that the benchmark for whether an automated prediction is “unwarranted” is whether it strays from the proportion of a group in the larger population. That is, if a prediction matches the current crime rate, it is still unjust! Even so, many who are grappling with how to enact ethical practices in this arena still use the crime rate as the default measure of whether an algorithm is predicting fairly, when that very measure is a byproduct of ongoing regimes of selective policing and punishment.