Algorithms
Algorithms are most helpful when applied to problems where there is not only a large history of past
cases to learn from but also a clear outcome that can be measured, since measuring the outcome
concretely is a necessary prerequisite to predicting. But a prediction algorithm, on its own, will
focus relentlessly on predicting the outcome you provide as accurately as possible at the expense of
everything else. This creates a danger: if you care about other outcomes too, they will be ignored. So
even if the algorithm does well on the outcome you told it to focus on, it may do worse on the other
outcomes you care about but didn’t tell it to predict.
This concern came up repeatedly in our own work on bail decisions. We trained our algorithms to
predict the overall crime rate for the defendents eligible for bail. Such an algorithm treats every
crime as equal. But what if judges (not unreasonably) put disproportionate weight on whether a
defendant engages in a very serious violent crime like murder, rape, or robbery? It might look like
the algorithm’s predictions leads to “better outcomes” when we look at overall rates of crime. But
the algorithm’s release rule might actually be doing worse than the judges with respect to serious
violent crimes specifically. The possibility of this happening doesn’t mean algorithms can’t still be
useful. In bail, it turns out that different forms of crime are correlated enough so that an algorithm
trained on just one type of crime winds up out-predicting judges on almost every measure of
criminality we could construct, including violent crime. The point is that the outcome you select for
your algorithm will define it. So you need to think carefully about what that outcome is and what
else it might be leaving out.
