Every time a scientist shares code that is incomplete, undocumented, or simply broken, the work behind a published finding becomes effectively unrepeatable. This project will measure how widespread that problem is across UK academia, using large-scale data analysis to assess the quality of code linked to research papers and identify the structural reasons why many researchers fail to follow open-source best practices. The problem matters because computational methods now underpin research from AI to the humanities, yet the reproducibility crisis is worsened by inconsistent coding. Current evidence is limited and contradictory, drawn mostly from small manual studies or surveys of researchers’ attitudes. This study will fill that gap with a comprehensive, data-driven picture of how code quality affects research integrity and future productivity. If successful, the research will provide concrete benchmarks and policy recommendations—for funders, universities, and journals—on where to intervene to improve code sharing. That could make published findings more trustworthy and reusable, quietly strengthening the infrastructure of science itself without requiring any change in how the public interacts with research.
View original technical description
Computational methods are increasingly pivotal in modern research, spanning fields from artificial intelligence to the humanities. These methods enable impactful science by opening new interdisciplinary areas and enhancing empirical robustness. However, the shift towards quantitative research introduces challenges of research integrity and reproducibility, particularly due to the variability in the availability and quality of shared code. Researchers are increasingly learning programming languages to take advantage of these new methods. Code-based approaches offer greater flexibility, access to advanced techniques, and the ability to handle large-scale data and modelling. Programming is theoretically better suited for sharing, reproducing, and scrutinising research through collaboration and versioning software. While results can be reproduced and documented according to standards, the quality of code varies significantly across academia. Our project aims to use a combination of data driven and qualitative methods to investigate how programming best practices could transform academia. Adhering to best practices for code sharing and reproducibility could bring benefits to science akin to the advent of the modern publication system, facilitating transparency, enabling extensive peer review, and promoting method dissemination and reuse. Conversely, inconsistent coding practices and incomplete repositories could hinder the potential of computational methods. We will use large-scale analysis of open data, combined with select qualitative methods, to identify code associated with research projects and publications, developing indicators for code quality based on open-source best practices. Our analysis will focus on three key aspects: The impact of code quality on future research productivity. The effects of current practices on research integrity. The structural factors associated with the use or neglect of open-source software practices. These questions have received limited attention in existing literature, particularly from large data-driven studies. Most current evidence is limited and contradictory and comes from manual studies or surveys of researchers' attitudes towards open-source code. Our comprehensive analysis aims to inform future policy in two ways. First, we will estimate the gap between the promise and reality of open-source practices and their impact on scientific activity. Second, we will identify factors where policy changes could improve the landscape. Additionally, we will establish indicators to measure outcomes, providing benchmarks for future policy experiments.
Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.
Is something wrong? Let us know