Performance of ethnic minority versus White doctors in the MRCGP assessment 2016–2021: a cross-sectional study

Background Differential attainment has previously been suggested as being due to subjective bias because of racial discrimination in clinical skills assessments. Aim To investigate differential attainment in all UK general practice licensing tests comparing ethnic minority with White doctors. Design and setting Observational study of doctors in GP specialty training in the UK. Method Data were analysed from doctors’ selection in 2016 to the end of GP training, linking selection, licensing, and demographic data to develop multivariable logistic regression models. Predictors of pass rates were identified for each assessment. Results A total of 3429 doctors entering GP specialty training in 2016 were included, with doctors of different sex (female 63.81% versus male 36.19%), ethnic group (White British 53.95%, minority ethnic 43.04%, and mixed 3.01%), country of primary medical qualification (UK 76.76% versus non-UK 23.24%), and declared disability (disability declared 11.98% versus not declared 88.02%). Multi-Specialty Recruitment Assessment (MSRA) scores were highly predictive for GP training end-point assessments, including the Applied Knowledge Test (AKT), Clinical Skills Assessment (CSA), Recorded Consultation Assessment (RCA), and Workplace-Based Assessment (WPBA) and Annual Review of Competency Progression (ARCP). Ethnic minority doctors did significantly better compared with White British doctors in the AKT (odds ratio [OR] 2.05, 95% confidence interval [CI] = 1.03 to 4.10, P = 0.042). There were no significant differences on other assessments: CSA (OR 0.72, 95% CI = 0.43 to 1.20, P = 0.201), RCA (OR 0.48, 95% CI = 0.18 to 1.32, P = 0.156), or WPBA—ARCP (OR 0.70, 95% CI = 0.49 to 1.01, P = 0.057). Conclusion Ethnic background did not reduce the chance of passing GP licensing tests once sex, place of primary medical qualification, declared disability, and MSRA scores were accounted for.


INTRODUCTION
The role of doctors' ethnic group in differential attainment in the UK Membership of the Royal College of General Practitioners (MRCGP) licensing assessments is a continuing concern, 1 and causes are poorly understood. 2 A study by Esmail and Roberts suggested, despite lack of supportive evidence, that 'subjective bias due to racial discrimination in the clinical skills assessment may be a cause of failure for UK trained candidates and international medical graduates'. 3 Despite a judicial review in 2014 4 and subsequent narrative review finding no evidence of racial discrimination, 5 there has been ongoing focus from some commentators on addressing unconscious bias, changing assessments, or addressing other unproven factors such as self-efficacy, and inclusion and relationships with educators and peers. 6 The Royal College of General Practitioners (RCGP), General Medical Council (GMC), and Health Education England (HEE)responsible for licensing of GPs, medical regulation, and postgraduate education, respectively -in response to the judicial review, have undertaken initiatives related to assessment, training, and research, which are designed to address differential attainment.
These include the following: aligning the curriculum and assessments to GMC Excellence-by-design standards, which have fairness as a guiding principle; reviewing and revising assessments where possible to reflect the UK patient population and reduce potential for differential attainment, including stakeholder engagement, pilots, and equality impact assessments for new or revised assessments; recruiting examiners and exam advisers from underrepresented groups and providing mandatory equality, diversity, and inclusion training; developing educational events and resources to support trainers and candidates including those who have failed exams in exam preparation; reviewing results, reports, guidance, and feedback to minimise risk of unconscious bias and to meet accepted guidelines for those with disabilities; and finally prioritising research into differential attainment. [7][8][9] Confounding factors implicated in differential attainment by doctors' ethnic group include age, sex, and place of primary medical qualification; although these have been included in previous studies of differential attainment, other factors, also related to ethnic group, such as declared disability or performance at selection into GP Abstract Background Differential attainment has previously been suggested as being due to subjective bias because of racial discrimination in clinical skills assessments.

Aim
To investigate differential attainment in all UK general practice licensing tests comparing ethnic minority with White doctors.

Design and setting
Observational study of doctors in GP specialty training in the UK.

Method
Data were analysed from doctors' selection in 2016 to the end of GP training, linking selection, licensing, and demographic data to develop multivariable logistic regression models. Predictors of pass rates were identified for each assessment. training, have less often been accounted for. [10][11][12] Complex educational and social factors may affect educational progress. 13 The potential contribution of these structural inequalities 14 is recognised in the term 'awarding' rather than 'attainment' gap. 15 These factors are rarely included in statistical models because data are lacking. 13 Performance at selection into specialty training, reflecting prior education and medical education, may affect endpoint attainment. 16 Selection for general practice training has involved the following three-stage process: 17 • stage 1, administrative: candidates provide proof of eligibility for UK specialty training, including proof of foundation-level competence;

Results
• stage 2, Multi-Speciality Recruitment Assessment (MRSA): a computer-based multiple-choice question examination, including both clinical problem-solving items and Situational Judgement Tests, developed and delivered by the National Recruitment Office as a shortlisting tool for many medical specialties, including general practice; and • stage 3, Selection Centre (SC): a GP-specific face-to-face assessment using objective structured clinical examination (OSCE)-style simulations and a written test for those scoring <575 on the MSRA. The SC was suspended in 2020 during the COVID-19 pandemic.
The MRCGP licensing test consists of the following three components: a computer-based Applied Knowledge Test (AKT); the Clinical Skills Assessment (CSA), a 13-station OSCE with role-players; 18 and Workplace-Based Assessment (WPBA), which informs an Annual Review of Competence Progression (ARCP) panel. Since 2020, the CSA was replaced by the Recorded Consultation Assessment (RCA), which uses 13 audio or video recordings of real patient consultations carried out and selected by candidates. Candidates are allowed up to five attempts at the AKT and the CSA or RCA, which includes four standard attempts and an exceptional fifth attempt, which most who request it are allowed.
Previous research has found that scores at selection into GP training were predictive of performance in the AKT and CSA. 18,19 No previous studies have explored differential attainment in WPBA-ARCP and evidence that differences in performance reflect prior academic performance is lacking. 6 This study aimed to investigate the extent of differential attainment by ethnic group in all components of the MRCGP, including the AKT, CSA, and WPBA-ARCP, while considering important potential confounders such as performance at selection into GP training, sex, disability, and place of primary medical qualification.

METHOD Design
A longitudinal design was employed, using retrospective data for doctors' performance from selection to the end of GP training, linking selection, licensing, and demographic data from doctors entering GP specialty training in 2016. The research question was as follows: is performance in the MRCGP (AKT, CSA, RCA, or WPBA-ARCP) different in ethnic minority versus White doctors? The objective was to investigate differences in performance in the MRCGP comparing ethnic minority with White doctors. The null hypothesis was that there was no difference in performance between ethnic minority and White doctors.
Setting, data collection, and processing All doctors entering UK GP specialty training in 2016 were included. They were followed up with all licensing test outcomes until the end of 2021.
MSRA and SC scores (available only for those scoring <575 on the MSRA) for doctors undertaking selection tests in 2016 were linked with their AKT, CSA, RCA, and WPBA-ARCP outcomes to 2021.

How this fits in
Differential attainment is widely found in undergraduate and postgraduate medical examinations. It has been suggested that subjective bias due to racial discrimination in clinical skills assessments may be a cause of examination failure for UK-trained ethnic minority candidates and international medical graduates. To the authors' knowledge, no previous study has examined differential attainment in all components of GP licensing assessments, including the Workplace-Based Assessment, considering scores at selection in GP specialty training. Ethnic background did not reduce the chance of passing GP licensing tests once sex, place of primary medical qualification, declared disability, and selection (Multi-Specialty Recruitment Assessment [MSRA]) scores were considered. Doctors admitted to GP specialty training, who are in the lowest MSRA score bands, may need additional support during training to maximise their chances of achieving licensing, regardless of their ethnic group or other demographic characteristics.
Individual candidate data provided by the GP National Recruitment Office, HEE, were linked with assessment outcomes and demographic data at the RCGP and transferred securely as a pseudonymised dataset, under a data-sharing agreement with the research team.
Individual candidates were assigned a unique (non-personally identifiable) number to link the various assessments to demographic data, including sex, ethnic group, country of graduation, and declared disability (specific learning difficulties and other physical disabilities), and assessment results, including overall scores, scores for assessment subdomains, and outcomes of pass (1)  Reasonable adjustments were provided for candidates with disabilities depending on their needs and requirements, which were based on a specialist assessment for written examinations and clinical assessments, including extra time (the standard is 25% additional time), a separate room for testing, and extra breaks. 20 Binary variables included the following: country of graduation (UK versus non-UK graduates), sex (male versus female), and declared disability (declared disability recorded versus no declaration of disability).
WPBAs are undertaken throughout the year and progress of the trainee is reviewed by a panel at their ARCP at the end of each academic year. Outcomes were categorised as 'standard' (for example, achieving progress and competencies at the expected rate or gaining all required competencies for completing training) or 'developmental' (for example, further development of specific competences required), and there is also the option of releasing the candidate from the training programme. The main outcome variables were pass (1) or fail (0) for the AKT, CSA, or RCA examinations, and presence of only standard ARCP outcomes (1) versus at least one developmental outcome or release from training (0).
MSRA scores were divided into 12 score bands and SC scores were divided into seven score bands, which were based on distribution of data and to achieve bands narrow enough to precisely identify candidates with differing performance.
It was estimated that a minimum sample size of 830 would be needed to see even a small effect size of 0.02 with five predictors, power 90%, and probability 0.05. 21

Statistical analysis
Descriptive statistics were used, indicating percentages of candidates passing each assessment and mean scores for CSA and RCA subdomains. Multivariable logistic regression models were used to determine the effect of ethnic group on licensing performance once sex, country of primary medical qualification, declared disability, and MSRA score bands were accounted for. Assumptions of no multicollinearity and no outliers were checked. Odds ratios (ORs), representing the odds that the outcome would occur given a predictor, compared with the odds of the outcome occurring in the absence of that predictor (that is, at baseline), and pseudo R 2 , representing the certainty with which the model can predict the dichotomous outcome (y = 0 or y = 1), were reported.    Table S1). There were no missing data for the variables of interest. Disabilities declared were chiefly specific learning difficulties (86.3% of all disabilities), but also included physical disability (1.6%), visual impairment (1.6%), hearing impairment (1.2%), and other disabilities (9.3%) (data not shown).

RESULTS
Pass rates were the highest for AKT, with 98.2% of candidates passing within the study period, followed by the CSA (92.4%) and RCA (85.8%). Pass rates were lowest for the RCA alone, but the number of possible attempts was lowest (three compared with five for AKT and CSA) and circumstances were different owing to its introduction during the COVID-19 pandemic. Raw pass rates at the first attempt were generally higher for White compared with mixed and ethnic minority candidates for the AKT (86.9%, 86.6%, and 61.6%), CSA (80.3%, 87.1%, and 66.4%), or RCA (95.5%, 88.2%, and 77.9%) (Supplementary Table S2).
MSRA score bands were the strongest predictors for all GP licensing outcomes at the 5-year point (AKT, CSA, RCA, and WPBA-ARCP). Lower SC score bands corresponded to poorer GP training outcomes but adding SC scores did not change the predictive validity of the MSRA. Therefore, the SC did not add further information to MSRA scores and were therefore not included in the logistic regression models.
Pass rates in AKT, CSA, or RCA and standard outcomes in the ARCP for ethnic minority doctors were no longer significantly different for White British doctors when MRSA scores and demographic factors, including sex, country of qualification, and declared disability, were considered. Conversely, ethnic minority doctors did significantly better compared with White British doctors in the AKT (OR 2.05, 95% CI = 1.03 to 4.10, P = 0.042) once these factors were taken into account, as seen in Table 1. There were no significant differences on the other assessments: CSA (OR 0.72, 95% CI = 0.43 to 1.20, P = 0.201), RCA (OR 0.48, 95% CI = 0.18 to 1.32, P = 0.156), or WPBA-ARCP (OR 0.70, 95% CI = 0.49 to 1.01, P = 0.057) (as seen in Tables 2-4).
Sex differences in performance were apparent in the CSA and WPBA-ARCP, with males doing significantly worse than females (Tables 2 and 4). International medical graduates (IMGs) performed significantly less well than UK-trained graduates in the CSA, RCA, and ARCP but not the AKT (Tables  1-4). Finally, candidates who declared a disability, most of whom stated they had a specific learning difficulty, performed significantly less well in the CSA and ARCP but not the AKT or RCA, although numbers included were small, particularly in the RCA.
White or ethnic minority IMGs had lower pass rates more pronounced in the CSA and in-training ARCP outcomes than UK doctors (Supplementary Figure S4). Logistic regression models accounting for sex, disability, and prior MSRA attainment with White UK doctors as comparators indicated that overseas-trained ethnic minority doctors performed significantly better on the AKT (OR 2.52, 95% CI = 1.03 to 6.16, P = 0.043) ( Table 5). Both White (OR 0.19, 95% CI = 0.07 to 0.48, P = 0.001) and ethnic minority (OR 0.15, 95% CI = 0.08 to 0.30, P<0.001) doctors not graduating in the UK performed significantly less well on the CSA, but this was not the case for ethnic minority doctors graduating in the UK (OR 0.55, 95% CI = 0.28 to 1.09, P = 0.086). Only ethnic minority non-UK doctors performed significantly less well on the RCA (OR 0.11, 95% CI = 0.03 to 0.45, P = 0.002). Being a White (OR 0.34, 95% CI = 0.18 to 0.62, P<0.001) or ethnic minority (OR 0.29, 95% CI = 0.19 to 0.43, P<0.001) IMG predicted a significantly lower likelihood of obtaining only standard ARCP outcomes, but this was not the case for ethnic minority UK graduates (OR 0.70, 95% CI = 0.49 to 1.01, P = 0.055). Detailed results can be seen in Table 5. All other groups had a poorer performance on all subdomains of the CSA and RCA compared with White UK graduates, but this was more pronounced in White and

Figure 1. Performance as indicated by mean scores for all subdomains of a) the Clinical Skills Assessment and b) Recorded Consultation Assessment.
ethnic minority IMGs on the interpersonal skills subdomain (Figure 1).

DISCUSSION Summary
Ethnic minority doctors performed no worse in GP licensing assessments when MSRA scores and demographic factors (sex, country of qualification, and declared disability) were considered. Ethnic minority doctors in general (OR 2.05, 95% CI = 1.03 to 4.10, P = 0.042) and non-UK ethnic minority doctors in particular (OR 2.52, 95% CI = 1.03 to 6.16, P = 0.043) were significantly more likely to pass the AKT compared with White British doctors once these factors were taken into account.

Strengths and limitations
There were high rates of completeness for outcome and demographic data. This study followed the 2016 cohort to 2021 as it was anticipated that most participants undergoing 'standard' 3-year full-time or extended GP training programmes would by then have attempted licensing assessments. However, not all participants who were unsuccessful in licensing tests would have had the opportunity to take them the permitted four times. For AKT and CSA this number was small (only 6% of candidates), but it involved all participants for the RCA who could only attempt this assessment three times by the end of the study.
Candidates on training extensions, maternity leave, and so on may have successfully completed training after the study end. The absence of significant differences for IMGs and those with declared disabilities in the RCA may have been owing to the smaller numbers of candidates who were able to take this assessment.
The analysis simplified categories of doctors who had qualified in the UK or overseas, those from ethnic minorities, or those with disabilities together, which does not take into account differences by medical school, country of primary qualification, ethnic group, or nature of disability. This was partly because the study did not have data on subcategories but also because increasing the number of categories would have provided groups that were too small for analysis.

Comparison with existing literature
Previous studies of differential attainment have included sex, place of primary medical qualification, and declared disability as covariates, 10,11 but selection scores, although known to predict licensing outcomes, have rarely been included in analyses. 18 In this study, MSRA was one of the main factors influencing outcomes, with ethnic group ceasing to be a significant predictor for any endpoint assessment when MSRA scores were taken into account.
A study examining the predictive value of selection tests showed strong correlations with educational supervisor rating at 1 year and performance in the AKT and CSA. 18 Another study combining MSRA and SC scores to investigate prediction of performance in AKT and CSA in one deanery found good prediction for the combined score. 19 In the present study, IMGs were just as likely to pass AKT but less likely to pass CSA or RCA or achieve only standard ARCP outcomes once MSRA scores and other demographic factors were accounted for. The explanations for differential attainment in IMGs are complex and multiple but are likely owing to 'difference in training experience and other cultural factors between candidates trained in the UK and abroad'. 3 These may include differences at recruitment to medical school or postgraduate training, during training and performance at assessments, cultural barriers (language difficulties, lack of understanding of cultural norms, and bias against seeking support or additional training), more limited professional networks (lack of mentorship or peer support), social challenges (poor work-life balance, separation from family, and lack of social support outside the work setting), and psychological difficulties (stress, anxiety, and burnout). 13,22 Another factor affecting performance of non-UK graduates in clinical licensing tests may be differences in initial medical training, where a doctor-centred rather than patientcentred approach to consulting may be taught and learnt. 23 These results suggest that prior attainment and training experience are the main factors driving the successful performance on the various licensing assessments.
Overall, the findings indicate that prior attainment and a primary medical qualification outside the UK were the main factors influencing performance on licensing assessments. A previous study examining CSA performance in ethnic minority doctors graduating in the UK or overseas indicated that IMGs had the poorest performance. 3 The present study showed that prior attainment as recorded by MSRA scores, having a disability, being male, and graduating outside of the UK as a White or ethnic minority person

Funding
This study was funded by Health Education England. The study funder had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication. The views expressed are those of the authors and not necessarily those of the funder.

Ethical approval
Ethical approval was received from the University of Lincoln Human Ethics Committee (reference: 2002_3645).

Data
Data will be available from the corresponding author on reasonable request with appropriate ethics and governance approvals.

Provenance
Freely submitted; externally peer reviewed.
were all significant predictors of lower pass rates on the CSA, but being of ethnic minority background and graduating in the UK was not. It is extremely unlikely that all these findings are due to subjective bias.
A more plausible interpretation would be that the CSA assesses certain skills that pose more difficulties for certain candidates including males, those with a declared disability, and those of different ethnic backgrounds graduating outside the UK. Moreover, the previous study did not compare performance directly, but ran independent logistic regression models and considered the differences in likelihood of failing the CSA. 3 The present study used the White UK graduate category as baseline and directly compared performance with all other ethnic groups on all licensing assessments. Importantly, the MSRA scores were used as an indicator of prior attainment, rather than AKT scores, which form part of the licensing assessments, with some candidates taking the CSA before the AKT. In the analysis, even accounting for AKT scores, the ethnic minority British group did not have a significantly poorer performance on the CSA.
Lastly, it is important to consider that the licensing assessments are based on a well-established pedagogy, and they are internationally recognised and used, which may indicate that candidates who fail are simply not ready for independent general practice.

Implications for research and practice
Differential attainment is present throughout the educational and training journey, and the correlation across longitudinal assessments, termed the academic backbone by McManus and colleagues, 16 is also seen in selection and licensing assessments.
The finding that ethnic status had no significant effect on performance at licensing assessments once selection scores and other demographic factors were accounted for suggests that, rather than the explanation being related to ethnic group, the reason for these differences, at least at licensing, is owing to differences on entering GP training rather than examiner bias, poorer relationships with educators and peers, or environment. 24 GP trainees should receive educational support appropriate to their needs, whatever their ethnic group or other demographic characteristics, particularly doctors admitted to training with low selection scores who may need additional support to maximise their chances of successful licensing. The present findings do not conflict with evidence that differential attainment by ethnic group, and potential factors associated with it, may be operating at medical school 15 or even earlier in the educational journey.
A previous systematic review, suggesting areas for support for doctors with protected characteristics, identified several factors that could influence differential attainment including learning and working environment, training experience and progression, learning and knowledge, and behavioural factors such as motivation and affect. 25 Interventions aimed at addressing differential attainment should consider these factors, but more rigorous research is needed to investigate the effect of possible interventions to address underperformance.
Educational interventions focusing on candidates who fail one component of the assessment, although these may be helpful, 26 could be replaced by support offered at the outset of training, for example, ensuring fairness in allocation of more sought after training practices and rotations, and enhanced educational provision, such as the Scottish Trainee Enhanced Programme (STEP). 27 This should be available to those who have been found to have low scores at selection and others who feel they may benefit, for example, IMGs, although this will need to be done carefully and communicated sensitively to avoid stigmatising this group of trainees.
More robust intervention development and stronger evaluation designs would add to the quality of evidence. Future studies should use larger datasets to explore differences by medical school, country of primary qualification, ethnic group, or nature of disability in greater detail and other factors that contribute to variation in performance at entry into specialty training for general practice. In addition, further studies should explore the relationship of entry standards to licensing outcomes and the factors that add value during training and improve subsequent performance.
In conclusion, ethnic background did not reduce the chance of passing GP licensing tests once sex, place of primary medical qualification, declared disability, and MSRA scores were considered. Comparing candidate scores by ethnic group creates a false impression of differential attainment, which should be addressed by routinely taking these other factors into account. Doctors admitted to GP specialty training in the lowest MSRA score bands may need additional support during training to maximise their chances of achieving licensing, regardless of their ethnic group or other demographic characteristics.