A self-driving car cannot simply be trained to win at all costs—it must learn to merge into traffic alongside human drivers who make mistakes, hesitate, and change their minds. Current AI systems handle competitive scenarios well, where there is a clear win condition. But cooperative settings—robots working alongside factory workers, or autonomous vehicles sharing roads with human drivers—lack that clear objective. The problem is that today’s cooperative AI relies on large amounts of human data to learn, which limits how far it can scale. Without that data, the AI has no way to pick out the rare human-compatible solution from a vast pool of incompatible ones. The researcher hypothesises that humans rely on an intuitive concept of a “good coordination solution”—one that is simple, symmetric, and easy to adapt to. This project aims to formalise that concept mathematically, then build AI agents that can discover such solutions on their own, without needing human data. The resulting methods could scale to complex real-world coordination problems that current AI cannot handle, unlocking applications such as autonomous industrial robots that work safely alongside people.
View original technical description
The goal of this proposal is to develop artificial intelligence (AI) agents, that can support and collaborate with humans in complex, real-world settings. These include, for example, industrial or service robots that can work in teams with humans and self-driving cars that interact smoothly with other traffic participants in mixed-autonomy settings. A fundamental issue is that, unlike the scalable solutions for competitive settings, current approaches for cooperative ones rely on human data and are thus limited in their scalability. Unfortunately, scaling compute to remove the need for human data is challenging in these settings. Without the well-defined objective present in zero-sum settings, it requires finding one of the few solutions that is human-compatible in a pool that also contains combinatorially many human-incompatible ones. My hypothesis is that humans have a well-defined concept of a 'good coordination solution' and to a great extent rely on this concept to solve coordination problems, i.e. when they have to work with others but cannot pre-agree on a strategy. Generally speaking, a good solution in such scenarios is one that is simple, symmetric, and therefore easy to adapt to. To move towards a formalisation and implementation of this intuitive idea, I will show how general purpose coordination policies can be efficiently discovered in complex settings using iteratively learned state-abstractions which implement simplicity and symmetry constraints. I will then robustify these policies to human sub-optimality using novel algorithms that gradually relax the constraints via online adaptation or small amounts of real-world human data. This project will result in new methods that can scale to complex human-AI coordination problems beyond the reach of the current state of the art. It will also develop a new theory that sets the scene for fundamental progress on human-AI coordination and unlocks crucial application areas, such autonomous industrial robots.
Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.
Is something wrong? Let us know