Assembling Certainty: Expert Knowledges and Machine Learning in Civilian Casualty Investigations

Assembling Certainty examines the ethical and epistemic implications of using Machine Learning (ML) in ‘open source investigations’ of civilian casualties in warzones. Combining backgrounds in software studies, digital sociology, and investigative practices, the project is led by Dr David Young (Project Lead, King’s College London) with Dr Josh Bowsher (Project Co-Lead, University of Sussex) and the casualty recording watchdog Airwars (Partner Organisation). Since 2014, Airwars has systematically documented civilian casualties arising from military airstrikes in conflicts, including Gaza, Syria and Ukraine, with the aim of holding responsible governments accountable. Over the past decade, open source intelligence (OSINT) has become an important framework for organisations like Airwars. To document civilian harm, Airwars pursues a necessarily deliberative process of assembling diverse (and sometimes contradictory) material posted on social media. However, the pace of commentary and challenges of verifying content means that using OSINT effectively is a growing challenge. Some organisations (New York Times, Forensic Architecture) have experimented with ML tools to analyse OSINT and verify civilian harm allegations, with some success. Such experiments also prompt us to re-examine how 'certainty' is assembled through novel human-machine knowledge practices, while critically exploring the consequent epistemic contingencies and gradations involved. Combining innovative practical and qualitative methods, this project will develop a ML toolkit with Airwars to explore its implications for their expert knowledges and specify guidance about its ethical use in future investigations. Working collaboratively, we ask: Q1. How does ML reconfigure the way investigators conceive of 'data' and 'sources'? Q2. How do established OSINT practices, expertise, and software interact during the process of designing/deploying an ML toolkit? Q3. Given ML inherently deals in probabilities, how might its use in investigations change our understanding of what constitutes ‘certainty’? The project will run for 20 months, organised around an autoethnographic approach to co-developing the ML toolkit. This will be scaffolded by qualitative interviews with the Airwars team: firstly, mapping their expertise, knowledge practices and ethical safeguards; and latterly, measuring the toolkit’s impact on their investigations process, noting possibilities, limits, and frictions. The project will produce: A ML toolkit (including, for instance, object recognition, image classification, semantic textual analysis) developed to support a specific investigation. PL-authored publication responding to Q2, documenting the development of the ML toolkit with Airwars. Two further publications co-authored by the PL and PCL responding to Q1 and Q3 above. A Guidance Document outlining considerations regarding when/how/why ML could be responsibly employed in casualty recording. A symposium consolidating a network of academics, journalists, and civilian-harm monitors involved with casualty reporting, acting as a launch event for the Guidance Document. The project has significant potential for social impact. The guidance document and symposium provide pathways to changing investigative practices regarding the use of ML and OSINT not only within Airwars, but also through their existing partnerships with humanitarian and news organisations. It also presents an important developmental opportunity for the PL to expand his research portfolio, develop skills through mentorship and training, grow a research community, and gain experience leading an expert multi-disciplinary team.