Synthetic Metascience: Tracing Artificial Intelligence-generated epistemic shifts in scientific research practice and cultures

Synthetic data - data generated using Artificial Intelligence (AI), designed to mimic real-world data characteristics or patterns without direct reference to real persons, objects, or events - has become increasingly prominent in discussions of science and technology innovation. A recent notable example is the UK government’s AI Action plan which specifically highlights the national importance of developing AI-generated synthetic datasets for scientific purposes. The increasing prominence of synthetic data in scientific research, where synthetic data is often framed as a solution to concerns around data privacy and model robustness, warrants deeper understanding of impacts. This timely interdisciplinary project will contribute critical theoretical and empirical knowledge about how synthetic datasets impact scientific research and innovation across the scientific ecosystem, exploring how synthetic data may potentially even redefine the role of the scientist. It will focus on the case study of biomedical research, a high-stakes area which interfaces closely with many scientific disciplines, to look at how AI methods and synthetic data are shaping scientific concepts and practice. The findings of this study will provide insights for the broader AI metascience community into how to foster rigorous, equitable scientific engagement with synthetic data. Long acknowledged issues in both AI and biomedicine, such as data bias and sparsity, make synthetic data a particularly promising proposition, with many of these issues anticipated to be resolvable through the generation and use of synthetic data. Meanwhile, as AI increasingly assumes exploratory and problem-solving functions, researchers become tasked with validating, refining, and contextualising machine-generated outputs. These developments raise critical questions about the authority and epistemic implications of AI-derived knowledge in comparison to traditional hypothesis-driven science, such as the impact of ground-truthing data gaps on veracity of synthetic data outputs. Evaluating science requires empirical and conceptual understanding of the nature of scientific roles and practice. However, synthetic data represents a paradigm shift which challenges deeply held norms of reasoning and evidence evaluation, distributing innovation between the human and machine. Responding to these challenges, this study will employ qualitative and empirical research methods to address key questions regarding how synthetic data reshapes epistemic practice and cultures within this case study domain. This research will address the following research questions; a) how does synthetic data mediate the role of the scientist, processes of problem formulation and the nature of scientific evaluation?, b) how do AI epistemic cultures redefine scientific concepts, such as scientific results and evaluation, relationships between inputs and outputs, and ideas of validity, reliability, and inference?, and c) what are the implications of this for building scientific cultures which employ synthetic data in a robust, open and conscientious way? Specifically, I will use interviews and interactive workshops in combination with document analysis of academic literature and other relevant writing, employing Reflexive Thematic Analysis to guide data analysis. Study findings will map how synthetic data are employed in practice, and how synthetic data mediates scientific practice. In doing so, this work will contribute crucial knowledge to the AI metascience community and inform governance and policy shaping engagement with AI methods in scientific.