Clinical trials that rely on patient questionnaires may be drawing flawed conclusions because the questionnaires themselves have never been properly tested for accuracy. Patient Reported Outcome Measures (PROMs)—the standardised questionnaires patients fill out about their symptoms, quality of life, or function—are used in thousands of trials to decide whether a new treatment works. Yet many PROMs have never been rigorously validated. If a questionnaire groups questions incorrectly, measures different things in different patient groups, or has too much random error, the trial’s results could be biased. This project will systematically review PROMs used in recent NIHR-funded trials, then re-analyse existing trial datasets using psychometric techniques to test whether measurement assumptions hold. Where they do not, the researcher will run sensitivity analyses to see if the trial’s conclusions change. If successful, this work will give funders and policymakers a practical checklist for selecting trustworthy PROMs and interpreting their results. For the public, it means greater confidence that a treatment shown to improve quality of life in a trial actually does so—and that the numbers behind that claim are not an artefact of a poorly designed questionnaire.
View original technical description
Background Patient Reported Outcome Measures (PROMs) are commonly used to measure outcomes in randomised controlled trials (RCTs). Not unreasonably, researchers, trialists and the users of research, often make assumptions about a PROM's measurement properties, including: Correct (unidimensional) grouping of items (questions) into scales (questionnaires) Measurement invariance between population subgroups (scores mean the same thing in different groups) Interval scaling (for scores purported to be continuous) Negligible measurement error Consistent relationship between score and measured trait, each time PROM is administered A meaningful target difference between groups PROMs are not all developed to the same standards, and many have not been fully validated with psychometric techniques that confirm these properties. Simulation studies have suggested that where measurement assumptions are violated, bias can occur. It is unclear how much this affects the trial's results or clinical conclusion. Aims To evaluate sources for measurement uncertainty in RCTs, use psychometric techniques to test and correct for erroneous measurement assumptions, determine the impact of these techniques on trial results, and develop guidance for funders, researchers, and policy makers. Objectives Perform a systematic review to evaluate the quality of PROMs used in NIHR-funded trials, as outlined by the COnsensus-based standards for the selection of health Measurement INstruments (COSMIN) guidelines. Test PROM measurement assumptions and perform sensitivity analyses in a series of existing RCT datasets to understand how these might affect the real-life trial results. Co-produce multistakeholder guidance on the use of psychometrics for PROM selection, psychometric sensitivity analyses and the interpretation of PROM data in RCTs. Methods I will conduct a systematic review of PROMs used in NIHR Health Technology Assessment-funded RCTs in the last 5 years to appraise each PROM against COMSIN criteria. This will help to identify common uncertainties surrounding the use of PROMs in RCTs that can in turn serve as targets for psychometric sensitivity analyses. Guided by the results of my systematic review, I will use existing trial datasets where a PROM has been used as a primary outcome measure to test measurement properties, including unidimensionality, measurement invariance, interval scaling, response shift, and measurement imprecision. I will perform sensitivity analyses to account for potential violations of measurement assumptions and compare results to those published. I will put these results to stakeholders in a Delphi study that will develop and refine a list of recommendations for accounting for PROM measurement assumptions in RCTs. This will be aimed at PROM developers, research funders, trialists and policymakers. Impact Where a trial's results are stable to tests of measurement assumptions, this will reassure readers that conclusions are robust, increasing the study's impact. Where this is not the case, it will highlight important areas of uncertainty and demonstrate techniques to address these. This could add granularity to existing trial results. Dissemination I will publish results in open access journals and present them. Communication teams and patient and public representatives will help dissemination to the broader public.
Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.
Is something wrong? Let us know