Candidates’ perceptions and experiences of the UK Recorded Consultation Assessment implemented during COVID-19: cross-sectional data linkage study

Problem

High stakes licensing exams have had to modify objective structured clinical examinations (OSCEs) to address the risks of the COVID-19 pandemic. The Recorded Consultation Assessment (RCA) pilot was rapidly introduced to replace the Clinical Skills Assessment (CSA) OSCE for UK general practice licensing during the pandemic. We aimed to evaluate candidate experiences and perceptions of the RCA and their relationship to exam performance.

Approach

We used a cross sectional online survey employing experience and demographic items, together with a free text response option. Statistical analysis with Stata 15.1 was used to undertake factor analysis of survey responses and to explore the association of candidate characteristics including gender, ethnicity, language, and place of primary medical qualification with factors (subscales) identified from the survey and with exam success. Qualitative thematic analysis supported by NVivo 12 was used to analyse free text responses. Binomial regression was conducted to estimate the association between RCA pass and candidate characteristics, number of attempts, questionnaire factors, consultation type, and trainer review as predictors.

Findings

Overall, 645 of 1551 (41.6%) candidates completed a questionnaire and 364 (23.5%) permitted linkage with RCA score and background data. Responders and non-responders were similar in exam performance, gender and declared disability but were significantly more likely to be UK graduates (UKG) or white compared with international medical (IMG) or ethnic minority graduates.

Responders were very positive about the digital platform and support resources. A small overall majority regarded the RCA as a fair assessment of clinical skills. A larger majority reported difficulty collecting, selecting, and presenting cases or felt rushed during recording.

Logistic regression showed that ethnicity (white vs minority ethnic: odds ratio [OR] 2.99,95% confidence interval [CI] 1.23, 7.30, p=0.016) , training (UK vs IMG: OR 6.88, 95% CI 2.79, 16.95, p<0.001), and English as first language (OR 5.11, 0% CI 2.08, 12.56, p<0.001) were associated with exam success but questionnaire subscales, type of consultation submitted, or extent of trainer review were not.

Consequences

This study shows the importance of evaluating novel assessments introduced during COVID-19. The Recorded Consultation Assessment was a feasible and broadly acceptable alternative to the Clinical Skills Assessment but had shortcomings perceived by candidates and showed areas for potential improvement.