Explaining differences in Membership of the Royal College of General Practitioners Applied Knowledge Test (MRCGP AKT) performance associated with candidate sex and ethnicity

Problem

There are well described differences in performance in the Membership of the Royal College of General Practitioners (MRCGP) Applied Knowledge Test (AKT) according to candidate sex, ethnicity and place of primary medical qualification (UK versus overseas). Little is known about why these differences exist although studies suggest they may largely relate to differences in educational experience. We aimed to investigate possible reasons for differences in performance by candidate sex, ethnicity and place of primary medical qualification, in particular exploring differences in undergraduate or postgraduate educational experience and psychological factors previously known to be associated with test performance.

Approach

We employed a cross sectional design linking self-completion questionnaire, test performance and routinely collected candidate attribute data. A self-administered computer delivered questionnaire was completed voluntarily by candidates undertaking the AKT. Questionnaire data were linked in consenting candidates with test performance and demographic data including attributes such as age, sex, ethnicity, and place of primary medical qualification. We asked questions on educational experience which may have contributed towards AKT performance such as: the quality of undergraduate, foundation year and specialty training for general practice; the strength of the trainee's relationships with their trainer, course organiser and peers; and the value of courses and peer study groups. We also asked questions on candidates’ perception of relevance and fairness of the test, their confidence in the test areas of clinical medicine, administration and data interpretation, and their estimated performance in the examination.

Findings

Overall 656 (of 1484) candidates completed questionnaires and consented to data linkage. Female candidates were more confident compared to males in data interpretation and statistics but less confident in practice administration. Male candidates felt they performed better in clinical medicine compared to females. Black and minority ethnic (BME) doctors reported a significantly worse relationship with their trainer compared to white doctors. BME candidates were less likely to feel they had adequate time or that the AKT was a fair test of knowledge compared to white candidates. In a multivariate analysis AKT performance was significantly lower (p <0.01) for candidates having more attempts, BME doctors and those trained abroad. Estimated score (insight) was a significant factor in performance whereas sex and educational experience were not. Undergraduate medical training overseas reduced all measures, but especially data interpretation and administration questions.

Consequences

We found differences in training experience, confidence and insight among candidates according to sex and ethnicity. Insight (estimated performance) was found to be an important factor linked to actual performance. Factors associated with performance require further study to understand how they might be addressed through improved assessment and training.