Turing AI Fellowship: Reinforcement Learning for Healthcare

In this Turing Artificial Intelligence Acceleration Fellowship, I will focus on artificial intelligence for medical treatments and therapies. I take the view that AI is a question on how to realise artificial systems that solve practical problems currently requiring human intelligence to solve, such as those solved by clinicians, nurses and therapists. Critical care is high risk and highly invasive environment caring for the sickest patients at greatest risk of death. Patients within this environment are highly monitored, enabling sudden changes in physiology to be attended to immediately. In addition, this monitoring requires a heavier staffing ratio (often 1:1 nursing; 1:8 medical) and variances in human factors and non-technical pressures (e.g. staffing, skill-mix, finances) leads to critical care delivery being disparate. AI in healthcare is a hard problem as, due to the diversity and variability of human nature, systems have to cope with unexpected circumstances when solving perceptual, reasoning or planning problems. Crucially, AI has two facets: Understanding from data, and Agency. While rapid strides have been made on learning from data, e.g. how to make medical diagnosis more precise and faster than human experts, there is little work on how to carry on after the diagnosis, e.g. which therapy and treatment to conduct. The latter requires agency and has seen fewer applications as it is a harder problem to solve. My clinical partners and I want to develop the required AI algorithms that can learn and distil the best plan of action to treat a specific patient, from the expert knowledge of clinicians. We will focus on an area of AI called RL that has been successful in enabling robots and self-driving cars to learn a form of autonomous agency. We want to transform these methods into the healthcare domain. This will require the development of new RL algorithms, able to efficiently understand the state of a patient from noisy and ambiguous hospital data. The system will not only learn to recommend interventions such as prescribing drugs and changing dosages as needed per patient but to make these recommendations in a manner that is meaningful to the clinical decision-makers and helps them make the best final decision on a course of action. The methods developed as part of this project can be used in different applications beyond healthcare. Many sectors within industry, such as aerospace, or energy, deal with similar bottlenecks. These are highly regulated environments, with great need for decisions making support, but a scarcity of highly skilled human experts. With sufficient data, our methods can be applied to these sectors as well, to distil the required human expertise and best practices from top experts, and use them to drive decision making all over the sector, for increased efficiency and safety.