Validation and Public Health Modelling of Risk Prediction Models for Kidney Cancer in UK Biobank

Talk Code: 
4B.6
Presenter: 
Hannah Harrison
Twitter: 
Co-authors: 
Hannah Harrison, Lisa Pennells, Angela Wood, Sabrina H. Rossi, Grant D. Stewart, Simon J. Griffin, Juliet A. Usher-Smith
Author institutions: 
1Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK 2Department of Surgery, University of Cambridge, Addenbrooke’s Hospital, Cambridge, UK

Problem

In the UK, kidney cancer is responsible for 4500 deaths annually. Although early detection is associated with improved survival rates, 25% of newly diagnosed kidney cancers are metastatic. One barrier to the introduction of a screening programme is the low population prevalence of kidney cancer. Population risk stratification could minimise harms to individuals and improve the efficiency of a screening programme. Stratification requires a model that accurately identifies individuals at high risk of undiagnosed kidney cancer. Although several models have been developed most have not been externally validated, and the benefits of incorporating them in a screening programme have not been assessed.

Approach

We identified phenotypic risk models in a recent systematic review and validated them in a large population cohort (UK Biobank) with 6-year follow-up. We assessed discrimination and calibration of the models for men, women and the whole cohort. We undertook a public health modelling analysis using the best performing models to estimate their accuracy in the UK population (individuals aged 40-70). We accounted for differences in demographics (age and sex) and kidney cancer incidence between the UK Biobank cohort and the general population, using ONS and CRUK data respectively. We compared the ability of the models to identify high-risk individuals for screening with simple age- and sex-based screening strategies.

Findings

We included seventeen studies (corresponding to 30 models) in the review. Eight models had reasonable discrimination (AUROC>0.62) in men, women and the mixed sex cohort. However, many of the models had poor calibration in the UK biobank cohort. Public health modelling demonstrated the accuracy of the best models over a range of thresholds (6-year risk: 0.1%-1.0%). At any particular risk threshold, the models performed very similarly. At all thresholds considered they showed a small improvement in ability to identify high-risk individuals compared to age- and sex- based screening. At a cut-off threshold of 0.4%, the best performing model screens 12.3% of the population and detects one case for every 180 individuals screened. Screening all men over the age of 60 (14.1% of the population) would detect one case for every 206 individuals screened. All of the models performed less well in women than men.

Consequences

This is the first comprehensive external validation of risk prediction models for kidney cancer. Five models showed both reasonable discrimination and good calibration in a UK-based population. The best-performing models could improve the efficiency of screening by similar amounts in a UK population, with the choice of model depending on the availability of data. However, very few people are predicted to have a 6-year risk higher than 1% and the models have worse performance in women. Future research may consider the potential benefits of adding biomarkers or genetic risk factors to phenotypic models.

Submitted by: 
Hannah Harrison
Funding acknowledgement: 
HH was supported by a National Institute of Health Research Methods Fellowship (RM-SR-2017-09-009) and is now supported by a National Institute of Health Research Development and Skills Enhancement Award (NIHR301182). SHR is supported by The Urology Foundation and a Cancer Research UK Clinical Research Fellowship. GDS’s work on this topic is funded by Kidney Cancer UK, The Urology Foundation, The Rosetrees Trust, Yorkshire Cancer Research and Cancer Research UK and supported by The Mark Foundation for Cancer Research, the Cancer Research UK Cambridge Centre [C9685/A25177] and NIHR Cambridge BRC. The University of Cambridge has received salary support in respect of SJG from the NHS in the East of England through the Clinical Academic Reserve. JUS was funded by a Cancer Research UK Prevention Fellowship (C55650/A21464). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.