Automated Systems
As we discussed before, automated systems are in the business of not simply “predicting” but producing crime. In Europe as elsewhere, already racialized and criminalized groups that try to exercise their newfound right to avoid data dragnets simply by opting out become doubly suspect: “What do you have to hide anyway?” (Recall the assumption of guilt for those who refused DNA collection by the United Kingdom’s Human Provenance Pilot Project, discussed in chapter 3.) In this way, data portability, like other forms of movement, is already delimited by race as a technology that constricts one’s ability to move freely.
Efforts to combat coded inequity cannot be limited to industry, nonprofit, and government actors, but must include community-based organizations that offer a vital set of counternarratives about the social and political dimensions of the New Jim Code. Too often, “inclusion” is limited to the visible representation of people of color and presumes that those who have been excluded want to be part of a tech future envisioned by others. As one reviewer put it, “Microsoft ‘Improves’ Racist Facial Recognition Software” – a win for inclusion! A loss for justice?
Abolitionist toolmaking must entail the democratization of data – both its design and its application. For example, the DiscoTech (“discovering technology”) model developed by the Detroit Digital Justice Coalition is a replicable approach that aims to demystify tech as a first step toward mobilizing community participation in questioning and shaping “data-driven” decisions that impact people’s lives.
Similarly, the Our Data Bodies (ODB) project retells stories of surveillance and data-based discrimination from the perspectives of those who are typically watched but not seen. The ODB team recently published a Digital Defense Playbook – an abolitionist tool based on in-depth interviews with community research participation in three US cities.70 The Playbook presents some of the strategies that individuals and organizations are using; but, in the spirit of Frederick Douglass’ admonishment about the upperground railroad, not everything that the team knows is exposed. Detroit-based digital justice activist Tawana Petty put it bluntly: “Let me be real, y’all gettin the Digital Defense Playbook, but we didn’t tell you all their strategies and we never will, because we want our community members to continue to survive and to thrive and so … the stuff that’s keepin’ them alive, we keepin’ to ourselves.”
If we come to terms with the fact that all data are necessarily partial and potentially biased, then we need long-term approaches that optimize for justice and equity. Timnit Gebru, computer scientist and founder of Black in AI, urges companies to give more information not just to users, but also to researchers. For any set of data, they can include “recommended usage, what the pitfalls are, how biased the data set is, etc.” She says: if “I’m just taking your off-the-shelf data set or off-the-shelf model and incorporating it into whatever I’m doing, at least I have some knowledge of what kinds of pitfalls there may be. Right now we’re in a place almost like the Wild West, where we don’t really have many standards [about] where we put out data sets.”
