A new software layer will force machine learning models to show their working, making their decisions transparent and verifiable. Current AI systems often operate as black boxes—they produce outputs without any clear explanation of how they reached a conclusion. This is a serious problem for safety-critical applications such as medical diagnosis, autonomous driving, or credit scoring, where regulators and users need to know that a decision is correct and fair. The European Union's proposed AI Act requires exactly this kind of accountability, but existing methods for checking AI behaviour are too slow and unreliable to be practical. The UNREAL project solves this by converting any machine learning model into a standardised format called a circuit. Once in this form, the model's reasoning can be broken down into small, provably correct steps. These steps can then be reassembled to verify complex behaviours—such as whether a model will always avoid a dangerous action—quickly and with mathematical certainty. The project will also produce a software toolchain that can take already-deployed models and either compile them directly into circuits or convert them into simpler versions that retain most of their accuracy while becoming verifiable. If successful, UNREAL will give regulators and engineers a practical way to certify AI systems before they are deployed, turning regulatory requirements from an obstacle into a solvable engineering problem.
View original technical description
Probabilistic reasoning is the ideal candidate to enable and foster the trustworthy deployment and execution of artificial intelligence (AI) and machine learning (ML) systems---a foundational requirement specified in the regulation that the European Union (EU) recently proposed in the AI Act---as it allows to inspect the behavior of a ML system at deployment time in the presence of uncertainty. Unfortunately, probabilistic reasoning is actually often a bottleneck rather than an enabler. In fact, there is no systematic way to engineer efficiently and reliably reasoning routines for different complex behaviors of the many ML models we have already deployed. At the same time, ML and AI practitioners have no principled and general methodology to design reliable ML models ex novo. This project will close these gaps by providing a UNified REAsoning Layer (UNREAL). UNREAL will deliver a theoretical framework under which different ML formalisms can be abstracted in a unified computational representation: circuits. UNREAL will decompose the task of reasoning over the behavior of ML systems into smaller modular primitives over these circuits that will be provably reliable and efficient. Then, by composing them, UNREAL will automatically guarantee the design of efficient and reliable routines for complex reasoning tasks. At the same time, UNREAL will provide a software toolchain in which already learned and deployed ML systems can be compiled into circuits for fast and certifiably reliable execution, or be automatically ``reconverted'' into tractable surrogates that support reliable reasoning and approximately retain the original predictive performance. UNREAL will provide the theoretical and practical foundations to design, distill, and deploy reliable ML systems, and therefore realize a path towards implementing the EU regulation for trustworthy ML.
Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.
Is something wrong? Let us know