Implementation failure or intervention failure? Process evaluation findings from the 3D study

Problem

Interventions to address the multiple challenges of managing multi-morbidity in primary care are necessarily complex and may incorporate behaviour change and/or system change components, implemented in diverse environments. Intervention success or failure depends on multiple factors and contextual influences, making it difficult to evaluate fidelity and discriminate between implementation failure and intervention failure. A pragmatic cluster-randomised trial in 33 general practices in the UK evaluated a complex, patient-centred intervention to improve management of multimorbidity in general practice and patients’ quality of life, with a null primary outcome. A concurrent process evaluation assessed intervention fidelity and explored context, intervention implementation and participants’ response to the intervention.

Approach

Case study methodology was supplemented by cross-trial surveys and capture of electronic data in a mixed methods design. Nineteen clinicians and seven administrators in selected practices were interviewed. Twenty-four clinicians were observed delivering the intervention and 38 patients attended interviews and focus groups. Surveys at the beginning and end of the trial captured usual care and attitudes to the intervention. Delivery of intervention components was recorded in an electronic template. Analysis of different data types was integrated and was finalised before the trial results were known.

Findings

Clinicians welcomed the intervention concept but practices experienced implementation difficulties due to staff loss and pressure on appointments. Only 50% patients received two multimorbidity reviews in 12 months as intended but 75% received at least one. These did not always replace usual care reviews, unnecessarily adding to the number of appointments. Delivery of patient-centred components in the reviews varied in quality and individual components were not always completed but reviews successfully elicited a full range of patients’ health concerns. Goal-setting to address patients’ agendas was less well performed and comments about training suggested this could have been improved. The potential value of pharmacist medication reviews was not fully realised. These findings led to a prediction of no change in the primary outcome of quality of life. Patients valued being asked about their concerns and receiving extended, comprehensive reviews, leading to a prediction of improvement in their perceptions of patient-centred care in intervention practices. Trial results confirmed both predictions.The intervention was incompletely delivered but nevertheless achieved improved patient-centredness. Multiple factors might have contributed to the null primary outcome, but we conclude the null result is more likely due to implementation failure than intervention failure, in a complex and challenging context.

Consequences

Implementation might improve if the intervention becomes mainstream and embedded in practice systems, and if the training and support for practices was focused on identified weaknesses. If the patient-centred multimorbidity reviews were delivered over a longer period, clinicians could better adjust to the novel format and health outcomes might change with more time.