The ethics of good data research (NESTA UK)

Data science techniques such as machine learning have been enthusiastically embraced by many for their potential to address some of society’s biggest problems. While much of this excitement is merited, novel data technologies can create new ethical issues, even when their developers and users are well-intentioned. Not everyone will agree on what a good outcome looks like.

The picture is further complicated by the structural injustices of racism, sexism, ableism and imperialism, all of which could be perpetuated or exacerbated by certain types of research if not given proper consideration. When it comes to high-stakes domains such as social care, healthcare or education, it’s essential that we approach the development and use of these technologies with a careful, critical lens.

In the light of this data environment, we have been navigating challenging questions about the aims and impact of Nesta’s work.

Over the past two years, we have experimented with cutting-edge ideas in the ethics of innovation in general and data science in particular. We have tested out tools and techniques from generative critique (an approach that builds critique into project processes, helping teams to interrogate theories of change and explore alternative courses of action) to datasheets for datasets (a tool for documenting the backstory of a dataset, including its potential biases and limitations).

Over the past year, our work has focused on questions such as how we can move beyond framing ethics as a constraint on creativity. We have also been thinking about what constitutes the ideal version of one’s professional role (eg, what does it mean to be a good data scientist?). The answers to these questions affect the data science community’s collective capacity to act on big, seemingly intractable societal problems.

One of our biggest challenges is finding an actionable approach to addressing structural injustice in research and innovation projects. This gap exists for a number of reasons. Structural issues can feel big, overwhelming and complicated, making it difficult to know where to start or how one’s actions can have an impact. In addition, there is a longstanding belief in the research and innovation field that its processes should tackle technical problems but avoid values-based goals such as social justice.

Drawing from theoretical and applied work in moral philosophy, critical data studies and science and technology studies (STS), we have developed an approach which we think can help research and innovation teams make structural injustice tractable.

The Role Ideal Moral Imagination (RI-MI) framework is built around two main components. First, the Role-Ideal model helps individuals craft ideal versions of their professional roles and visions for what they might achieve, as well as to identify and act in ways that push the boundaries of what’s expected of them (e.g. what does it mean to be a ‘good’ data scientist?). The second component is the Moral Imagination framework: a tool for creative, values-based decision-making at the project level. We developed the RI-MI framework in collaboration with the Data Analytics Practice at Nesta, but we believe it has the potential for application in other disciplines too.

Many of the challenges that Nesta’s missions aim to address are shaped by structural conditions – factors such as race and socioeconomic background affect access to good health, education and other outcomes. Our framework can help research and innovation teams to see this structure and their role within it."

There are six main steps to the RI-MI framework

1. Critically reflect (and read up) on the relationship between data science and structural injustice, explore the range of theories and visions for how to solve the problem(s) you are working on and consider how your organisational context shapes your work.
2. (Re)define role-ideals by developing the ‘ideal’ version of your role, building from insight gained in the previous step. Ask what resources and powers do I have in my role? What would it mean to do my role well? How can my role be used to challenge structural injustice rather than reinforcing it?
3. Identify boundary-pushing opportunities in the form of specific actions and behaviours that will help achieve your role-ideal, drawing on resources and powers, as well as positive exemplars (others who have successfully pushed the boundaries of their role).
4. Exercise moral imagination at the project level, enacting the direction set through your role-ideal. Apply the following scaffold to project decisions: identify the goal or task to be completed; examine the underlying values operationalised in the project approach; explore the landscape of legitimate stakeholders and how they are to be involved and; identify the range of alternative approaches that might maximise different values or the interests of different stakeholder groups.
5. Practise moral criticism by setting an accountability structure around your role-ideal and develop routines and processes for providing encouragement, (appropriate and constructive) pressure and feedback at the team level.

Share and openly debate and practise RI-MI in the open by documenting and sharing role-ideal and moral imagination processes and outputs, in order to enable feedback, improvement and contestation.

Many of the challenges that Nesta’s missions aim to address are shaped by structural conditions – factors such as race and socioeconomic background affect access to good health, education and other outcomes. The RI-MI framework can help research and innovation teams to see this structure and their role within it and to use this understanding to explore, assess and debate different approaches at the project level. We look forward to reporting on our progress as we apply the RI-MI framework in our data analytics work and beyond.

We are presenting a paper From ideals to action in AI for social good: Introducing the RI-MI framework at the Ethics of Socially Disruptive Technologies conference in October, and will aim to publish a version of this paper in due course. If you are interested in discussing the RI-MI framework further, please get in touch: chantale.tippett@nesta.org.uk, juliet.ollard@nesta.org.uk, camille.stengel@nesta.org.uk.