Factors associated with differential performance in the MRCGP Clinical Skills Assessment: cross-sectional study

Problem

Differential performance in medical licensing exams is a continuing concern for candidates, exam bodies and professional organisations. Previous studies investigating doctors' performance in medical exams suggest that personal, educational, cultural or psychological factors may be important influences leading to differential attainment by candidate attributes such as ethnicity, gender or qualification overseas in International Medical Graduates. The ‘Fair Pathways’ report suggested that poorer relationships between trainee doctors and their seniors, trainers or peers, negative expectations and sociocultural factors may be important sources of difficulty underlying differential attainment. The extent to which such factors contribute to differential attainment is largely unexplored and unknown. We aimed to examine the relative contribution of these factors to differential attainment in a national licensing clinical skills assessment.

Approach

We used a self-administered questionnaire to candidates which they completed immediately after taking the Clinical Skills Assessment (CSA) of the UK Membership of the Royal College General Practitioners (MRCGP) licensing exam. Candidates volunteered to participate and agreed to link their questionnaire, exam and demographic data for analysis. We analysed data using SPSS version 25 to produce descriptive statistics, comparisons and multiple regression to model factors independently associated with passing or failing the CSA.

Findings

Overall 44.6% (209 of 469) CSA candidates completed the questionnaire between 30 November and 6 December 2018, and consented to data linkage. Multiple logistic regression showed that being older (odds ratio [OR] 0.86, 95% confidence interval ([CI] 0.75 – 0.99, p=0.04), male (vs female OR 0.22, 95% CI 0.06 - 0.79, P=0.02), of minority ethnic status (vs white British OR 0.04, 95% CI 0.01- 0.31, P=0.002) or having a negative vs positive relationship with their trainer (OR 0.11, 95% CI 0.01 – 0.88, p=0.04) were associated with a reduced chance of passing the CSA. There was no association with passing or failing in International Medical Graduates (IMG vs UK trained doctors OR 0.38, 95% CI 0.08 -1.91, p=0.24), those who preferred the English language (vs those who preferred another language OR 1.20, 95% 0.13 - 10.72, p=0.87) or candidates’ estimated score (OR 1.07, 95% CI 0.99 - 1.16, p=0.07). Candidates who did not declare a specific learning difficulty vs those who did had a greater chance of passing the CSA (OR 11.49, 95% CI 1.27 – 103.51, p=0.03).

Consequences

CSA failure was associated with older age, male sex, minority ethnic status, specific learning difficulty, and a negative relationship with the candidate’s trainer but not with being an International Medical Graduate, preferring English language or estimated score. Further research is needed to understand why specific learning difficulties and a negative relationship with a trainer are associated with failure, and to evaluate interventions designed to reduce differential CSA performance.