Facial recognition cameras and AI-driven welfare assessments are already changing how people behave and how the state treats them, often in ways no one intended. This project is the first long-term effort to map those unintended consequences—specifically the indirect harms to human rights and social equality that ripple out from AI systems. The core problem is that governments increasingly use AI to make decisions about healthcare, child protection, criminal justice, and social welfare, and those decisions feed into each other, potentially locking people into discriminatory cycles. To get ahead of this, the researchers are building an open-source computer model that simulates an interconnected AI decision-making ecosystem. It will let civil society groups, advocates, and policymakers test how a welfare decision might cascade into a criminal justice outcome years later—before those systems are built. If successful, the model could become a standard tool for anticipating stratification and chilling effects, rather than reacting to them after the damage is done. The project also rethinks international human rights law to account for surveillance’s subtle erosion of identity and political participation, working with Amnesty International and the UN Special Rapporteur on Freedom of Assembly.
View original technical description
This is the first long-term project to examine AI’s ‘unintended consequences’: the indirect (but impactful) harm associated with AI deployments. The objective is to develop a more holistic understanding of AI’s overall impacts and harms, thereby facilitating more effective human rights-based decision-making, to inform legal and policy approaches to the development and deployment of AI. The overall project is built around two overarching research strands. First, is an empirical examination into AI’s ‘unintended consequences’, specifically unintended human rights and societal harms. Phase 1 of the FLF focused on understanding the ‘chilling effects’ of AI-linked surveillance (such as facial recognition technology); if, and how, individuals modify their behaviour in light of surveillance, impacting their ability to freely develop their identity and to participate in political processes. Significant empirical work has been undertaken, and Phase 2 will further develop this analysis. The next phase of research aims to get ahead of the curve by anticipating the likely direction of travel with respect to governmental uses of AI in decision-making processes. This centres around the development of a computer model to simulate a future interconnected and interdependent AI decision-making ecosystem (where outputs from AI-assisted decisions, say relating to social welfare, are used to update an individuals’ profile, which then feeds into a range of further AI-assisted decisions – relating to health care, child protection, criminal justice, etc. – with these outputs in turn informing additional future decisions). The concern is that such an ecosystem may exacerbate existing discrimination, and may – over time – contribute to stratification within society. The testing made possible by this model provides a rare opportunity to anticipate and plan for, rather than react to, AI developments. This work will be conducted in partnership with Open Lab Athens, Lighthouse Reports and Amnesty International. The computer model will be made available open source, so that organisations around the world can adapt it to replicate local systems, and to inform advocacy and policy approaches. Second, this knowledge is used to rethink existing approaches to international human rights law, to adapt to the AI era. This adaptation is key to the development of appropriate legal and policy responses, and is explicitly interdisciplinary. A key focus is on better incorporating surveillance-related chilling effects, and protections for the free development of individuals’ identity (including the concept of ‘identity as a whole’), into decision-making processes. This up-to-date approach to human rights law is centred around evidence-based assessments of potential utility and potential harm. Lighthouse Reports and Amnesty International (independently/in collaboration) will collaborate on investigations into existing AI systems, and the ‘AI ecosystem’ model, and develop linked reporting and advocacy. The open source and modifiable nature of the computer model will facilitate impact, as it can be used by civil society and other actors to conduct their own research, investigations, and advocacy. This pursues a specific social justice objective, by making cutting-edge technology accessible. The continued deployment of biometric surveillance technologies (such as facial recognition) and ever-increasing digital surveillance, ensures that the project’s chilling effects research remains strategically relevant. Work to secure impact in this area will continue with project partner, the UN Special Rapporteur on Freedom of Assembly. This project is delivering ground-breaking interdisciplinary research while also contributing to the development of national policy and international standards adopted by the United Nations.
Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.
Is something wrong? Let us know