The role of missing stage and chance in performance indicators for cancer stage at diagnosis: Cross-sectional analysis of population-based data for general practices, local authorities and care commissioning organisations in England

Problem

The imperative to improve cancer outcomes and to reduce inequalities, alongside rapid improvements in the availability of highly complete information on stage at diagnosis, have led to public reporting of stage at diagnosis distributions for CCGs and local authorities. Stage indicators have also been included in the CCG Quality Premium pay-for-performance scheme. However, the extent to which these indicators reflect real organisational differences rather than differences in stage completeness and chance variation is unclear.

Approach

We used population-based data on patients diagnosed with the 10 cancers included in the CCG Quality Premium indicator for early stage cancer. We examined the association between the proportion of patients within a CCG diagnosed at early stage (defined as TNM stage 1 or 2) and the proportion of patients with missing stage. We did this calculating the early stage proportion using only staged cancers and again using all cancers but treating missing stage as advanced. We then estimated the Spearman-Brown (organisation-level) reliability of the indicators for CCGs, local authorities and general practices based on only cancers with recorded stage and annual data (thus quantifying the role of chance). Finally we assessed probable misclassification rates for the CCG Quality Premium targets.

Findings

There was no association between proportion early stage and proportion missing stage when restricting to stage cancers whereas a strong negative association was seen when treating missing stage as advanced. Restricting our analysis to staged tumours we found strong evidence (p<0.001) of variation in stage at diagnosis across general practices, CCGs or local authorities. Despite the variation, all indicators based on one year of data had poor reliability. To meet the recommended reliability level of 0.7 would require reporting periods of 29 years or more for general practices and 2-3 years for CCGs and local authorities. Based on these findings we would expect 30 of 211 CCGs to be misclassified on the CCG Quality Premium indicator when considered in terms of their long-term underlying performance with only 56% of those hitting the target having an underlying performance above target (PPV).

Consequences

Our findings suggest that cancer stage at diagnosis indicators currently used are dominated by variations in missing stage. Restricting data to tumours with known stage results in indicators for CCGs and local authorities in England that lack reliability and the current pay-for-performance scheme needs to be re-thought. Stage at diagnosis indicators are highly unsuitable for classifying performance of general practices.