Can we use Serious Adverse Events to assess representativeness of randomised controlled trials? An observational analysis using aggregate and individual-level data from clinical trials and routine healthcare data

Conference:

SAPC ASM 2022 - UCLan

Talk Code:

3D.6

Presenter:

Peter Hanlon

Twitter:

@PHanlon17

Co-authors:

Peter Hanlon, Elaine Butterly, Anoop SV Shah, Laurie J Hannigan, Sarah H Wild, Bruce Guthrie, Frances S Mair, Sofia Dias, Nicky J Welton, David A McAllister

Author institutions:

Institute for Health and Wellbeing University of Glasgow, London School of Hygiene and Tropical Medicine, Norwegian Institute of Public Health, Usher Institute University of Edinburgh, Centre for Reviews and Dissemination University of York, Population Health Sciences University of Bristol

Problem

Randomised trials (trials) provide causal estimates of treatment efficacy, but there are concerns that trial findings may not be applicable to under-represented patients such as older people and those with multimorbidity and frailty. However, assessing the representativeness of trials to patient populations is complex and inherently subjective. We aimed to explore (i) the representativeness of trials by comparing Serious Adverse Events (SAE) in trials with those expected based on hospitalisation and death rates for people with the same index condition in routine care, (ii) whether multimorbidity counts predict SAEs in trials and in routine care, and (iii) whether there are residual differences between observed and expected SAE rates after accounting for multimorbidity.

Approach

We performed an observational analysis of individual and aggregate-level drug trial data for 21 index conditions compared to population-based routine healthcare data (routine care). Trials identified from clinicaltrials.gov. Aggregate data from 483 trials (n=636,267). Individual participant data (IPD) from 125 trials (n=122,069). Routine care comparison from linked primary care and hospital data from SAIL databank (n=2.3M). Our outcome of interest was incident SAEs. SAEs are routinely reported in trials. In routine care, SAEs were based on hospitalisations and deaths (which are SAEs by definition). We compared trial SAEs from aggregate trial data to expected SAEs based on age/sex standardised routine care populations with the same index condition. Using IPD, we assessed the relationship between multimorbidity count and SAEs in both trials and routine care, and assessed the impact on the observed/expected SAE ratio additionally accounting for multimorbidity.

Findings

For 12/21 index conditions the pooled observed/expected SAE ratio was <1, indicating fewer SAEs in trial participants than in routine care. A further 6/21 had point estimates <1 but the 95% CI included the null. The median pooled estimate of observed/expected SAE ratio was 0.60 (95% CI 0.56-0.65; COPD) and the interquartile range was 0.44 (0.34-0.55; Parkinson’s disease) to 0.88 (0.59-1.33; inflammatory bowel disease). Higher multimorbidity count was associated with SAEs across all index conditions in both routine care and trials. For all trials, the observed/expected SAE ratio moved closer to 1 after additionally accounting for multimorbidity count, but it nonetheless remained below 1 for most.

Consequences

Trial participants experience fewer SAEs than expected based on their age, sex and index condition. This suggests lack of trial representativeness, which has implications for trial applicability and estimating net treatment benefits. The difference between observed and expected SAEs is only partially explained by differences in multimorbidity count. Age/sex standardised observed/expected SAE ratios offer clinicians and guideline developers an objective, readily calculable, metric of trial representativeness, which may help guide assessment of the applicability of trial findings to routine care populations in whom multimorbidity and frailty are common.

Submitted by:

Peter Hanlon

Funding acknowledgement:

David McAllister is funded via an Intermediate Clinical Fellowship and Beit Fellowship from the Wellcome Trust (Grant reference - 201492/Z/16/Z). Peter Hanlon is funded through a Clinical Research Training Fellowship from the Medical Research Council (Grant reference: MR/S021949/1).