Self-driving cars, medical diagnosis tools, and automated benefit decisions are already making choices that affect people's lives, but there is no reliable way to prove these systems deserve our trust. This research builds a new regulatory and software engineering framework that lets autonomous systems, their developers, and their regulators evolve together, rather than treating regulation as a final hurdle after development is complete. The team—combining computer scientists, legal scholars, and ethicists—will create practical tools that generate and manage evidence of trustworthy behaviour as systems are continuously updated. They will also run real-world case studies, taking autonomous systems through regulatory processes with industry partners, to see what actually works. If successful, this could change how everything from energy grids to medical diagnostics is governed: instead of static safety checks that are outdated by the time a system ships, regulators and developers would share a living evidence base that adapts as the system learns and changes. The project does not promise to solve every trust problem, but it aims to replace the current ad-hoc approach with a repeatable, legally-grounded methodology.
View original technical description
How can we trust autonomous computer-based systems? Autonomous means "independent and having the power to make your own decisions". This proposal tackles the issue of trusting autonomous systems (AS) by building: experience of regulatory structure and practice, notions of cause, responsibility and liability, and tools to create evidence of trustworthiness into modern development practice. Modern development practice includes continuous integration and continuous delivery. These practices allow continuous gathering of operational experience, its amplification through the use of simulators, and the folding of that experience into development decisions. This, combined with notions of anticipatory regulation and incremental trust building form the basis for new practice in the development of autonomous systems where regulation, systems, and evidence of dependable behaviour co-evolve incrementally to support our trust in systems. This proposal is in consortium with a multi-disciplinary team from Edinburgh, Heriot-Watt, Glasgow, KCL, Nottingham and Sussex, bringing together computer science and AI specialists, legal scholars, AI ethicists, as well as experts in science and technology studies and design ethnography. Together, we present a novel software engineering and governance methodology that includes: 1) New frameworks that help bridge gaps between legal and ethical principles (including emerging questions around privacy, fairness, accountability and transparency) and an autonomous systems design process that entails rapid iterations driven by emerging technologies (including, e.g. machine learning in-the-loop decision making systems) 2) New tools for an ecosystem of regulators, developers and trusted third parties to address not only functionality or correctness (the focus of many other Nodes) but also questions of how systems fail, and how one can manage evidence associated with this to facilitate better governance. 3) Evidence base from full-cycle case studies of taking AS through regulatory processes, as experienced by our partners, to facilitate policy discussion regarding reflexive regulation practices.
Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.
Is something wrong? Let us know