09 November 2017

Motherboard knows best?

If decisions in future courtrooms are made by algorithms, how do people know they are fair?

If decisions in future courtrooms are made by algorithms, how do people know they are fair?


by Tebello Quotsokoane

If there’s anything that history has taught us, it is that people are not impartial. Racism, sexism and hetero-patriarchy were at one point or another codified in our institutions, organisations and communities. To quote one social activist, “prejudice seems to be in the air we breathe.” It is because of these realities that people, especially marginalised people, remain critical of institutions and structures. We know that human decisions are affected by implicit and explicit biases, not just in history, but even today.

Studies have shown over and over again that humans make judgements based on faulty intuition. People are dissuaded from providing dissenting views due to the fear of upsetting group cohesion. Human judgement is plagued by stereotyping and prejudice – the challenger accident is an example of the poor decisions that groupthink often leads to. It is as a direct answer to these faults of human judgement that the fields of evidence-based decision making, behavioural economics and behavioural psychology are borne.

The rise of algorithms may seem a compelling way to avoid the suboptimal decisions that humans sometimes make. Algorithmic decision-making tools use large amounts of data to come to conclusions and inform recommendations. In 2010, Andrew McAfee heralded algorithmic decision making tools as the future of decision making in a Harvard Business Review article. Applications of algorithmic decision-making in the fields of radiology and predictive medicine are already bringing better patient outcomes. Algorithms have also gained wide traction in marketing, policing and fraud detection, with mixed results.

One might expect, then, that algorithms would be seen as more objective arbiters in disputes, or more impartial recruiters for jobs. But they are not. Researchers have observed that people rely on human judgement, even when informed about the shortcomings of human decision making. These observations will be critical for governments as the public sector looks to use artificial intelligence (AI) in improving public service delivery. People are not particularly alarmed by the application of technology in the form of online platforms like turbo-tax or dashboards. The problem usually arises when algorithms are used to inform judgements which have a significant impact on citizens’ lives, such as sentencing criminals or deciding who should receive welfare payments. Machines are becoming increasingly intelligent and capable of making informed and often accurate judgement, but it would appear that humans still trust fellow humans for some realms of decision-making.

We may not always understand the inner thought processes of a human, but when a person makes an error you can ask them why and perhaps even challenge the decision in a formal appeal process. Machines, by contrast, make decisions that are often out of the purview of both government workers and ordinary citizens. This opacity may create mistrust in the outcomes of algorithmic decision-making. Human involvement in decision-making can feel fairer, because people are given the right to participate in the process, and to understand the basis on which a decision is made.

Governments should think about how to ensure developers of algorithmic systems remain open about how these tools operate, and that developers are held accountable for the outcomes of their algorithmically-informed decisions. This is especially true in cases where an AI system is so complex that even those with a total view into it are unable to describe its failures and successes. How and whether governments address these questions is likely to determine how successful AI implementation in government will be, and whether it is perceived as legitimate by the citizens it affects.


Tebello Quotsokoane worked as a research intern at Oxford Insights in summer 2017. 


More insights

21 April 2017

Why Government is ready for AI

12 July 2017

Five levels of AI in public service

26 July 2017

Making it personal: civil service and morality

10 August 2017

AI: Is a robot assistant going to steal your job?

19 September 2017

AI and legitimacy: government in the age of the machine

06 October 2017

More Than The Trees Are Worth? Intangibles, Decision-Making, and the Meares Island Logging Conflict

16 October 2017

The UK Government’s AI review: what’s missing?

23 October 2017

Why unconference? #Reimagine2017

03 November 2017

AI: the ultimate intern

23 November 2017

Beyond driverless cars: our take on the UK’s Autumn Budget 2017

05 December 2017

Why Black people don’t start businesses (and how more inclusive innovation could make a difference)

06 December 2017

“The things that make me interesting cannot be digitised”: leadership lessons from the Drucker Forum

23 January 2018

Want to get serious about artificial intelligence? You’ll need an AI strategy

15 February 2018

Economic disruption and runaway AI: what can governments do?

26 April 2018

Ranking governments on AI – it’s time to act

08 May 2018

AI in the UK: are we ‘ready, willing and able’?

24 May 2018

Mexico leads Latin America as one of the first ten countries in the world to launch an artificial intelligence strategy

05 July 2018

Beyond borders: talking at TEDxLondon

13 July 2018

Is the UK ready, willing and able for AI? The Government responds to the Lords’ report

17 July 2018

Suspending or shaping the AI policy frontier: has Germany become part of the AI strategy fallacy?

27 July 2018

From open data to artificial intelligence: the next frontier in anti-corruption

01 August 2018

Why every city needs to take action on AI

09 August 2018

When good intentions go bad: the role of technology in terrorist content online

26 September 2018

Actions speak louder than words: the role of technology in combating terrorist content online

08 February 2019

More than STEM: how teaching human specialties will help prepare kids for AI

02 May 2019

Should we be scared of artificial intelligence?

04 June 2019

Ethics and AI: a crash course

25 July 2019

Dear Boris

01 August 2019

AI: more than human?

06 August 2019

Towards Synthetic Reality: When DeepFakes meet AR/VR

19 September 2019

Predictive Analytics, Public Services and Poverty

10 January 2020

To tackle regional inequality, AI strategies need to go local

20 April 2020

Workshops in an age of COVID and lockdown

10 September 2020

Will automation accelerate what coronavirus started?

10 September 2020

Promoting gender equality and social inclusion through public procurement

21 September 2020

The Social Dilemma: A failed attempt to land a punch on Big Tech

20 October 2020

Data and Power: AI and Development in the Global South

23 December 2020

The ‘Creepiness Test’: When should we worry that AI is making decisions for us?

13 June 2022

Data promises to support climate action. Is it a double-edged sword?

30 September 2022

Towards a human-centred vision for public services: Human-Centred Public Services Index

06 October 2022

Why You Should Know and Care About Algorithmic Transparency

26 October 2022

Harnessing data for the public good: What can governments do?

09 December 2022

Behind the scenes of the Government AI Readiness Index

06 February 2023

Reflections on the Intel® AI for Youth Program

01 May 2023

Canada’s AI Policy: Leading the way in ethics, innovation, and talent

15 May 2023

Day in the life series: Giulia, Consultant

15 May 2023

Day in the life series: Emma, Consultant

17 May 2023

Day in the life series: Kirsty, Head of Programmes

18 May 2023

Day in the life series: Sully, Partnerships Associate/Consultant

19 May 2023

LLMs in Government: Brainstorming Applications

23 May 2023

Bahrain: Becoming a regional R&D Hub

30 May 2023

Driving AI adoption in the public sector: Uruguay’s efforts on capacity-building, trust, and AI ethics

07 June 2023

Jordan’s AI policy journey: Bridging vision and implementation

12 June 2023

Response to the UK’s Global Summit on AI Safety

20 June 2023

 Unlocking the economic potential of AI: Tajikistan’s plans to become more AI-ready

11 July 2023

Government transparency and anti-corruption standards: Reflections from the EITI Global Conference in Dakar, Senegal

31 August 2023

What is quantum technology and why should policymakers care about it?

21 September 2023

Practical tools for designers in government looking to avoid ethical AI nightmares

23 October 2023

Collective Intelligence: exploring ‘wicked problems’ in National Security

23 October 2023

Exploring the concepts of digital twin, digital shadow, and digital model

30 October 2023

How to hire privileged white men

09 November 2023

Inclusive consensus building: Reflections from day 4 of AI Fringe

13 November 2023

AI for Climate Change: Can AI help us improve our home’s energy efficiency?

14 November 2023

Navigating the AI summit boom: Initial reflections

20 November 2023

AI for Climate Change: Improving home energy efficiency by retrofitting

24 November 2023

Will AI kill us all?

27 November 2023

AI for Climate Change: Preventing and predicting wildfires 

28 November 2023

Service Design in Government 2023: conference reflections

04 December 2023

AI for Climate Change: Using artificial and indigenous Intelligence to fight climate change

06 December 2023

Release: 2023 Government AI Readiness Index reveals which governments are most prepared to use AI

11 December 2023

AI for Climate Change: AI for flood adaptation plans and disaster relief

18 December 2023

AI for Climate Change: Managing floods using AI Early Warning Systems