Governments are beginning to use AI to decide which scientific studies inform policy, but no one has yet measured whether AI picks different evidence than humans do. This matters because the way policymakers find and use scientific evidence has remained largely unchanged for decades. AI systems are now being adopted by governments to speed up evidence identification, yet there is no empirical data on whether these systems recommend substantially different research than current human-driven processes. The gap could either amplify existing biases in evidence selection or correct blind spots in human-curated work. The researcher will compare evidence historically cited in policy documents against what an AI system would have recommended for the same policy questions. Using a novel AI tool developed during the researcher’s PhD, in partnership with Elsevier, the project will analyse over two million policy documents across the United Nations Sustainable Development Goals. If successful, the research will produce a quantitative map of where AI and human evidence selection diverge, and identify researchers whose work is highly policy-relevant but currently overlooked. This could help universities spot talented researchers that current human-driven processes miss, and help governments understand which policy domains are most vulnerable to disruption as AI adoption accelerates.
View original technical description
Artificial Intelligence (AI) systems will fundamentally transform how scientific evidence is accessed by policy makers. As governments increasingly adopt AI-powered systems for evidence identification, conventional pathways for connecting science to policy will be disrupted. This represents a major shift in the science-policy interface that has been essentially unchanged for decades. While research on general AI ethics and societal impacts has grown rapidly, a significant empirical gap exists in understanding precisely how AI will alter which scientific evidence informs governmental decisions. This project addresses a fundamental question at the intersection of AI, metascience, and evidence-based policymaking: to what extent and in what ways does AI-sourced evidence differ from human sourced evidence in policy contexts? As governments transition toward AI-mediated evidence ecosystems, we need empirical analysis of whether AI systems recommend substantially different scientific evidence than current human-driven processes. These differences could either amplify existing limitations in evidence selection or potentially correct blind spots in human-curated evidence. I seek to conduct a large-scale, data-driven comparison between human-selected and AI-selected scientific evidence in policymaking. This analysis will create counterfactual comparisons between evidence historically cited in policy documents and what would have been recommended by AI for identical policy questions. I will quantify systematic differences in evidence selection patterns across the United Nations Sustainable Development Goals (SDGs) and governmental bodies, while measuring shifts in researcher networks between those currently cited in policy versus those identified by AI systems. To achieve these objectives, I will leverage a novel AI recommendation system developed during my doctoral research. In partnership with Elsevier, who will provide comprehensive metadata and access to their vast scientific database, I will apply this tool retrospectively to study differences between actual policy citations and AI-recommended research across the SDGs. I will do this in two stages, first by identifying a case study of systematic literature reviews conducted for a policy issue, and comparing what these reviews identified against what AI would identify, then based on lessons learnt, scale the analysis using Overton policy citation data to over 2 million policy documents worldwide spanning the SDGs. Finally, I will perform a meta-analysis examining patterns of AI vs human cited research. This research will deliver a quantitative analysis that measures the gap between AI-recommended and human-selected evidence across different SDGs, identifying which domains are most susceptible to disruption as AI systems are adopted. Second, a comprehensive dataset mapping the "invisible college" of researchers who produce highly policy-relevant work, but aren't currently recognised in policy citations. This will enable universities to identify and support overlooked talent that current human-driven processes miss.
Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.
Is something wrong? Let us know