How can we optimise the impact and effectiveness of national clinical audit programmes?

Problem

Audit and feedback aims to improve patient care by reviewing health care performance against explicit standards. It is widely used to monitor and improve patient care, including through National Clinical Audit programmes in the United Kingdom (UK). One example is the National Diabetes Audit which monitors primary care diabetes management and collates data from around 8,000 general practices in England. Audit and feedback is variably effective; conducting individual randomised trials to address the many unanswered questions about how to optimise its effectiveness would be relatively inefficient. We evaluated different modifications to feedback in an online experiment, as an efficient way of identifying leading candidates for further ‘real world’ evaluation. Even relatively small improvements to feedback’s effectiveness would have the potential for substantial population health impacts.

Approach

We conducted an innovative, fractional factorial screening experiment, randomising recipients of feedback from five UK national clinical audit programmes to different combinations of six feedback modifications through a web portal. Outcomes, assessed immediately after working through the online modifications, included intended enactment of audit standards (primary outcome), comprehension, user experience, and engagement.

Findings

We randomised 1241 participants (clinicians, managers and audit staff). During the response period, we detected suspicious activity associated with repeated (duplicate) participant completion. Our primary analysis population conservatively excluded 603 (48.6%) participants during the defined ‘contamination period’ and included 638 (51.4%) participants with 566 (45.6%) complete responses.

None of six modifications to feedback independently increased intended enactment of audit standards across the five audits. We observed both synergistic and antagonistic effects across all outcomes when feedback modifications were combined. For example, the most effective combination of modifications resulted in predicted intended enactment (on a scale of -3 to +3) of 2.40 (95% confidence interval 1.88, 2.93) versus 1.22 (0.72, 1.72) for the least effective combination in clinical participants in the National Diabetes Audit. Intended enactment for clinical participants was optimised by providing multimodal feedback, recommending specific actions, incorporating the ‘patient voice’, and minimising extraneous cognitive load. We also observed that the audit itself and whether recipients had a clinical role had a dominant influence on outcomes.

Consequences

None of six feedback modifications improved intended enactment of audit standards in isolation. However, we observed potentially important synergistic and antagonistic effects in various combinations of feedback modifications, audit programmes and recipients. This suggests that the design of feedback needs to explicitly consider how different features are likely to act together. The findings have implications for the design of national clinical audits, with the potential to enhance the effectiveness of audits used within UK primary care.