Systematic review of the applicability to real-world populations of randomised controlled trial evidence of treatments for physical health conditions

Problem

Guidelines rely on randomised controlled trials (RCTs) of treatments to create evidence-based recommendations to inform clinical practice. RCTs are the preferred method for this purpose because their design accounts for important sources of bias. However, previous reviews have shown that RCTs routinely have poorly justified eligibility criteria that systematically exclude important groups of the population, raising concern about the generalisability of findings to all patients seen in routine practice.

Approach

We conducted a systematic review of studies examining the proportion of people in research or clinical cohorts who would have been eligible for trials of treatments for physical health conditions. The review is registered at PROSPERO https://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42016042282 but in brief the Medline and Embase databases were systematically searched using a broad search strategy to identify studies examining the proportion of people with a condition who would have been eligible for an RCT of a treatment for that condition. All title/abstract/full-text eligibility screening and data extraction done independently by two reviewers. Risk of bias was assessed. Given study heterogeneity, analysis was descriptive/narrative.

Findings

17,124 titles and abstracts were screened, and 43 studies included examining the proportion of patients eligible for 291 trials of treatments for 25 conditions in comparison cohorts that included research registers, clinical registers of various kinds, and population surveys. The proportion of the comparison cohort eligible for the trials examined in descending order of the number of trials examined was: rheumatoid arthritis range 0.9-42.1% (median 8.4%, 62 trials), COPD 0-57.6% (median 15.7%, 53 trials), HIV 32-100% (median 58%, 32 trials), hypertension 0.4-98.4% (median 16.9%, 21 trials), heart failure 8-71% (median 27.2%, 21 trials), asthma 0-36% (median 4%, 17 trials), breast cancer 18.8-71.1% (median 43.4%, 13 trials), stroke/TIA 25.3-66.8% (median 28.8%, 10 trials), bronchiectasis 6.6-50.8% (median 18.9%, 9 trials), atrial fibrillation 28.9-67.7% (median 62.6%, 8 trials), type 2 diabetes 3.5-50.7% (median 18.3%, 7 trials), diabetic foot ulcers 0.4-70.2% (median 6.7%, 7 trials), venous ulcers 11.1-41.7% (median 16.4%, 7 trials), primary prevention of CVD 10.9-30.3% (4 trials), lung cancer (28.1-34.6% (3 trials), acute MI 9.4-15.5% (2 trials), dementia (7.5-13.5% (2 trials), angina 35.3-98.4% (2 trials), colon cancer 32.4-34.3% (2 trials), type 1 diabetes 4.4-12.5% (2 trials) with six conditions with one trial examined (from 26.9% eligible for irritable bowel syndrome to 65.3% for pressure ulcers). We will present data on variation by the nature of the comparison cohort and by studies at low and high risk of bias.

Consequences

For all but two conditions, median eligibility was <50% and usually <25%, most commonly because trials excluded older people, people with comorbidity and people with significant co-prescribing. Even assuming that treatment benefit is maintained in excluded populations, treatment harm is likely higher because of age, and drug-disease and drug-drug interactions. The benefit-harm balance in excluded populations is uncertain and better evidence is needed in older, more comorbid and more co-prescribed populations.