Variation and statistical reliability of publicly reported diagnostic activity indicators for cancer in primary care: A cross-sectional of routine data.
There are large international variations in cancer survival, even between high income countries. It has been suggested that such variations may, in part, reflect the presence/absence of ‘gate-keeping’ in primary care, leading to longer diagnostic intervals and poorer clinical outcomes for cancer patients. Recent public reporting initiatives in England highlight general practice variation in indicators of diagnostic or referral activity related to cancer. Better understanding of the size of variation and the reliability of practice-level estimates can help to optimise how this information is interpreted and used for quality improvement purposes.
We analysed ‘General Practice Profiles for Cancer’ data. Using appropriate mixed effect regression models we characterised the size of the underlying between practice variation in activity for different indicators, accounting for the role of chance and the age-sex profile of practice population. We further calculated the Spearman-Brown reliability of different indicators.
Indicators based on a single year of data exaggerate the differences between practices due to chance fluctuations. After accounting for the role of chance, there remained substantial variation between practices (typically up to 2-fold variation between the 75th and the 25th centiles of the distribution of practice scores, and up to 4-fold variation between the 90th and 10th centiles). The demographic (age and sex) make-up of practice populations explains some of this variation, variably for different indicators. Concerning indicator reliability, diagnostic outcome indicators relating to incident cancer cases (e.g. % of all cancer cases that were detected via emergency presentations) have reliability well below accepted minimum thresholds required. In contrast, diagnostic process indicators relating to broader populations of patients most of whom do not have cancer (e.g. rate of endoscopic investigations, or urgent ‘two-week-wait referrals’ for suspected cancer) have high or very high reliability.
Use of indicators of diagnostic activity in English general practices should principally focus on process indicators with adequate or high reliability (e.g. number of patients investigated) and not outcome indicators which are unreliably measured at practice level.