Individual participant data informed risk of bias assessments for randomised controlled trials in systematic reviews and meta-analyses

James X Sotiropoulos; Kylie E Hunter; Jannik Aagerup; Jonathan G Williams; Sol Libesman; Mason Aberoumand; Angie Barba; Rui Wang; Thomas D Love; Brittany J Johnson; Anna Lene Seidler

doi:10.1016/j.jclinepi.2025.111875

Individual participant data informed risk of bias assessments for randomised controlled trials in systematic reviews and meta-analyses

J Clin Epidemiol. 2025 Jun 26:111875. doi: 10.1016/j.jclinepi.2025.111875. Online ahead of print.

Authors

Affiliations

¹ NHMRC Clinical Trials Centre, Faculty of Medicine and Health, University of Sydney, Sydney, Australia; Faculty of Medicine and Health, University of New South Wales, Sydney, Australia.
² NHMRC Clinical Trials Centre, Faculty of Medicine and Health, University of Sydney, Sydney, Australia. Electronic address: kylie.hunter@sydney.edu.au.
³ NHMRC Clinical Trials Centre, Faculty of Medicine and Health, University of Sydney, Sydney, Australia; Department of Child and Adolescent Psychiatry, University Medical Centre Rostock, Rostock, Germany; German Center for Child and Adolescent Health (DZKJ), partner site Greifswald/Rostock, Rostock, Germany.
⁴ NHMRC Clinical Trials Centre, Faculty of Medicine and Health, University of Sydney, Sydney, Australia.
⁵ Flinders University, College of Nursing and Health Sciences, Caring Futures Institute, Tarntanya, Adelaide, South Australia, Australia.

PMID: 40581170
DOI: 10.1016/j.jclinepi.2025.111875

Abstract

Objective: In evidence synthesis, assessing risk of bias (ROB) of eligible studies is crucial to inform interpretation of findings. Standardised tools like Cochrane's ROB-1 or ROB-2 traditionally rely upon published information to inform assessments, but this is often incomplete or unclear. Availability of raw individual participant data (IPD) enables more in-depth assessments, yet, guidance on how to use IPD in ROB assessments is lacking. We aim to develop preliminary guidance on how to use IPD to inform ROB assessments of randomised controlled trials (RCTs) for three case studies.

Study design and setting: In stage 1, we reviewed relevant literature, consulted our networks, and drew on previous experience to compile items on how IPD may inform ROB assessment for each domain. We discussed feasibility and potential usefulness of each item with an international, interdisciplinary expert advisory group and developed preliminary guidance, which was piloted in two IPD meta-analyses (MAs) (65 RCTs) using ROB-1. In stage 2, the guide was adapted for ROB-2 and applied to another IPD-MA (34 RCTs). All assessments were conducted in duplicate by two independent reviewers. In stage 3, we conducted an evaluation workshop to further refine each item, and capture important lessons. To assess the impact of IPD-informed assessments, we compared them to existing ROB-1 assessments performed with published information alone for 33 trials.

Results: We identified 12 items across the ROB domains. IPD provided opportunities to enhance ROB assessments by enabling additional checks for selection bias (i.e., testing randomisation) and attrition bias (i.e., more granular assessment of incomplete data at various timepoints). We also identified domains for which availability of IPD enabled reduction of ROB, for example, by mitigating selective outcome reporting bias or by re-including excluded participants in intention-to-treat analyses. Applying IPD-informed assessments led to changes in ROB judgment in 25 out of 33 studies, most commonly, resolution of domains previously marked as 'unclear'.

Conclusion: Our preliminary guidance for IPD-informed ROB assessments may be applied in IPD-MAs to increase the accuracy of ROB assessments and in some cases reduce ROB to create a more reliable evidence base informing policy and practice.

Keywords: individual participant data; meta-analysis; risk of bias; systematic reviews.