The “Cohort Multiple Randomised Controlled Trial” design – does it work in practice?
Problem
Controlled Randomised Trials often struggle to recruit and there is interest in new designs that can more effectively recruit and retain patients and make trials more efficient. One innovation is the ‘cohort multiple randomised controlled trial’ (cmRCT).
The cmRCT design aims to make the trial consent procedure more like standard health care, where people are only asked to consent to treatments they are being offered and are not told about treatments they cannot access. Under this design a substantial cohort of patients is recruited, then followed up at regular intervals. To trial a new treatment, a random sample from all eligible participants is offered the treatment. The remaining eligible patients (those not offered the treatment) constitute the control arm but are not informed about the trial or about treatments they will not receive.
Advocates of the design claim significant advantages in recruitment, patient centredness, and efficiency. A number of patient cohorts and related cmRCTs have been established but very few have yet reported and good evidence for these claims is lacking.
Approach
We established the CLASSIC cohort of 4,377 patients with long-term conditions and conducted a cmRCT (“PROTECTS”) of a telephone-based health coaching intervention. Baseline and six-monthly follow-up data was collected over a two-year period. Use of the cmRCT design raised many methodological and statistical challenges so far not addressed in the literature. In this paper we consider these issues, report how we tackled them within CLASSIC and PROTECTS, and their implications for the design, conduct and analysis of cmRCTs.
Findings
We discovered many challenges to the use of the cmRCT design in actual practice. Principal are issues around statistical power, sample size, and treatment effect estimation. We provide appropriate methods for power calculation and show that unless levels of participant eligibility and consent are substantial, the sample size requirements for a cmRCT may be impracticable large.
We also describe hitherto unidentified validity risks inherent in the design, such as sampling practices commonly applied in pragmatic trials which when applied to a cmRCT can result in selection bias, and validity issues related to the fixed data collection points that feature in cmRCTs.
Consequences
The cmRCT may offer advantages over conventional trial designs. However, a cmRCT requires appropriate power calculation, sampling and analysis procedures, else studies may be underpowered or subject to validity biases. Pilot studies are essential to estimate rates of eligibility and non-consent for the purposes of cmRCT sample size estimation.
It is important that trials using the cmRCT design publish sufficient detail to re-assure users of the research that the validity threats specific to this design have been appropriately addressed. Further methodological investigations are needed and cmRCT-specific CONSORT guidance may be required.