The standard method for reviewing evidence—relying on randomised controlled trials as the gold standard—can produce misleading conclusions when mechanistic evidence about *how* an intervention works is ignored. A philosopher and a team of experts from medicine, law, policy, and statistics are building a new review method called Interdisciplinary Systematic Review (ISR) that formally integrates mechanistic evidence with trial data, using the effectiveness of face masks as a test case. This matters because the current hierarchy of evidence, championed by the Evidence-Based Medicine movement, assumes RCTs can stand alone. The COVID-19 pandemic exposed this weakness: conclusions about mask efficacy based solely on trials proved untrustworthy. Without a rigorous way to combine mechanistic and trial evidence, policy decisions on everything from public health to policing risk being built on incomplete foundations. If ISR succeeds, it could change how evidence is synthesised across medicine, education, and criminal justice. Rather than relying on a single study design, reviewers would have a systematic method to weigh both statistical outcomes and causal mechanisms. This is primarily methodological research—it refines the tools for evaluating interventions rather than testing a specific treatment. But better evidence synthesis means more reliable guidance for clinicians, policymakers, and the public.
View original technical description
Goal The goal of this project is to develop a new approach to evidence review: Interdisciplinary Systematic Review (ISR). ISR will provide a way to integrate mechanistic evidence from across disciplines, alongside the studies that are the focus of orthodox systematic reviews. Background Synthesising research evidence is key to evaluating the effectiveness of interventions and to developing policy. The dominant approach in medical disciplines, championed by the Evidence-Based Medicine movement and Cochrane, appeals to an explicit evidence hierarchy in which randomized controlled trials (RCTs) are seen as having inherent advantages over other forms of evidence. Other disciplines and fields have begun to emulate this approach, producing 'evidence-based education', 'evidence-based management', 'evidence-based policing' and more. But the RCT design has significant weaknesses if used in the absence of mechanistic evidence—evidence about the causal mechanisms by which interventions produce effects. The assumption that RCT evidence can stand alone led to untrustworthy conclusions about the efficacy of masks during the COVID-19 pandemic. The lead applicant, a philosopher, has already published a preliminary outline of how mechanistic evidence might be combined at a philosophical level with RCT evidence. This project seeks to take that work forward with a team of experts drawn from multiple disciplines (including medicine, law, policy, statistics), using a review of the effectiveness of face masks as a worked example. Research questions How can mechanistic evidence be systematically and rigorously combined with RCTs and other designs to produce a more interdisciplinary and robust evidence base for evaluating intervention efficacy? What are the strengths and limitations of this new approach ('Interdisciplinary Systematic Review', ISR) compared with other mechanism-sensitive (theory-based) review methods such as realist or metanarrative review? Study design Interdisciplinary desk research to develop and apply a novel systematic review methodology, combining insights from analytic philosophy, Bayesian statistics and theory-based systematic review.
Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.
Is something wrong? Let us know