The predictive validity of selection for entry into postgraduate training in general practice: evidence from three longitudinal studies

Fiona Patterson; Filip Lievens; Máire Kerrin; Neil Munro; Bill Irish

doi:10.3399/bjgp13X674413

Published eLetters

eLetter submission guidelines

Submit a Response to This Article

Compose eLetter

Title *

Author Information

Contributors
First Name and Middle Initial * First or given name, e.g. 'Peter'. Last Name * Your last, or family, name, e.g. 'MacMoody'. Email Address * Your email address, e.g. higgs-boson@gmail.com Occupation * Your role and/or occupation, e.g. 'Orthopedic Surgeon'. Affiliation * Your organization or institution (if applicable), e.g. 'Royal Free Hospital'.

Statement of Competing Interests

Competing interests? *

Yes

Please describe the competing interests

CAPTCHA

This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.

Vertical Tabs

Jump to comment:

BJGP paper shows that GP Selection Centre has low utility
Richard Wakeford

Published on: 19 December 2013

Published on: (19 December 2013)

BJGP paper shows that GP Selection Centre has low utility
Richard Wakeford, Life Fellow
The recent paper by Patterson et al [1] reports the encouraging and considerable predictive validity of MCQ (CPS = 'clinical problem solving') and situational judgment test (SJT) selection tests for performance on the two MRCGP formal examination components.
The MCQ and the SJT correlate with the Applied Knowledge Test (AKT) at 0.85 and 0.69, respectively (range-corrected). From data provided in the paper, togeth...
Show More

The recent paper by Patterson et al [1] reports the encouraging and considerable predictive validity of MCQ (CPS = 'clinical problem solving') and situational judgment test (SJT) selection tests for performance on the two MRCGP formal examination components.
The MCQ and the SJT correlate with the Applied Knowledge Test (AKT) at 0.85 and 0.69, respectively (range-corrected). From data provided in the paper, together the two selection tests can be seen to predict 74-75% of AKT score variance. The two tests also correlate with the old-style Clinical Skills Assessment (CSA: 0.55, 0.57, respectively, range-corrected). Together they predict 37-38% of CSA Score variance. But the subsequent Selection Centre (SC), then a three station OSCE and correlating with the CSA at 0.41 (range corrected), is reported as only explaining an additional 2% of CSA score variance.
Given that a set of computer-delivered multi-choice tests will cost a hundred pounds or two per candidate and that the true cost of a selection centre will be at least 1000 pounds, the latter seems to provide relatively very poor value, especially with about 5000 candidates being short-listed for selection in 2012 [2]. Perhaps the 5M-plus pounds could be better used in supporting poorly-performing trainees?
Also, those of us who are responsible for devising OSCE assessments would be grateful to learn how one with three stations, single marked, can be devised such that its reliability (Cronbach's alpha) is 0.87, when considerably longer similar assessments give far lower reliability estimates - for example, MRCGP CSA (13 stations) 0.77 [3], and iMRCS Part B (8 station sub-tests) 0.68 - 0.72 and (10 station sub-tests) 0.76 - 0.78 [4].
References 1. Br J Gen Pract 2013; DOI:10.3399/bjgp13X674413 2. National Recruitment Office: Summary of the recruitment data by recruitment stage and deanery (2012). http://www.gprecruitment.org.uk/annualreports.html 3. MRCGP Statistics 2011-12: Annual Report on the AKT and CSA Assessments, page 41. http://www.rcgp.org.uk/gp-training-and-exams/mrcgp-exam- overview/mrcgp-annual-reports/mrcgp-annual-reports-2011-2012.aspx 4. Intercollegiate Committee for Basic Surgical Examinations: 2012/13 Annual Report, page 9. http://www.intercollegiatemrcs.org.uk/new/pdf/Annual_Report_12_13.pdf
Conflict of Interest:
None declared
Show Less

Competing Interests: None declared.