Verifying appraisal evidence using feedback from trained peers: views and experiences of Scottish GP appraisers

Paul Bowie; Niall Cameron; Ian Staples; Rhona McMillan; John McKay; Murray Lough

doi:10.3399/bjgp09X453521

Abstract

Background GP appraisal is currently considered inadequate because it lacks robustness. Objective assessment of appraisal evidence is needed to enable judgements on professional performance to be made.

Aim To determine GP appraisers' views of the acceptability, feasibility, and educational impact of external peer feedback received on three core appraisal activities undertaken as part of this study.

Design of study Independent peer review and cross-sectional postal questionnaire study.

Setting NHS Scotland.

Method One of three core appraisal activities (criterion audit, significant event analysis, or video of consultations) was undertaken by GP appraisers and subjected to peer review by trained colleagues. A follow-up postal questionnaire elicited participants' views on the potential acceptability, feasibility, and educational impact of this approach.

Results Of 164 appraisers, 80 agreed to participate; 67/80 (84%) submitted one of three appraisal materials for peer review and returned completed questionnaires. For significant event analyses (n = 44), most responders believed the peer feedback method was feasible (100%) and fair (92.5%) and would add value to appraisal (95.5%). Peer feedback on criterion audits (n = 15) was believed to be acceptable and fair (93.3%) and it was thought it would be a useful educational tool (100%). Completing a consultation video (n = 8) was perceived to be feasible as part of normal general practice (n = 5). It was unanimously agreed that assessment of videos by peers has educational impact and would help improve appraisal.

Conclusion This group of GP appraisers strongly supported the role of external and independent feedback by trained peers as one approach to strengthening the existing appraisal process.

INTRODUCTION

Appraisal for GPs in NHS Scotland was introduced in 2003.1 This ‘non-judgemental’ process helps GPs to reflect on their working practices with a peer — a trained GP appraiser colleague. One of five core educational activities (Table 1) — each linked to the principles of Good Medical Practice,2 — is discussed in depth annually. Although appraisal has gained acceptance,3^–5 the benefits and impact are limited for many GPs.6^,7 When introduced, it was envisaged that the combined evidence from five appraisals would inform a proposed system of medical revalidation in the UK.8 The recent white paper makes clear, however, that the existing appraisal model is inadequate for this purpose because it lacks robustness and objectivity.9 Addressing this will require objective assessment of evidence to be included to allow judgements on professional performance.10

View this table:

Table 1

Core categories for GP appraisal in Scotland and alignment with the principles of the General Medical Council's Good Medical Practice.2

In the west of Scotland, a well-established educational model aims to provide GPs with informed, objective, confidential, and independent feedback from trained peers on the quality of three core appraisal activities: videoed consultations, criterion-based audit, and significant event analysis (SEA).11^–13 The model is coordinated by NHS Education for Scotland (NES) and has been used on a voluntary basis by GPs as part of regional arrangements for continuing professional development. Although the concept of ‘peer review’ is well established in medicine, the novel aspect of this model is that it appears to be the only one reported in the literature that utilises GP peers who are specifically trained to review and provide developmental feedback on the educational activities outlined. To date, around 40% of west of Scotland GP principals have submitted at least one piece of work for peer review since its inception.

The underlying principles of the model are based on an adaptation of cognitive continuum theory.14 This framework aids understanding of the thinking used in performing a range of tasks. The aim is to improve the quality of reflection on particular tasks. How this is done is described in one of six ‘modes of practice’ ranging from the highly structured scientific experiment (mode 1) to intuitive judgements (mode 6). Peer review sits between modes 4 (system-aided judgement) and 5 (peer-aided judgement) and is designed to minimise the probability of a mode 6 judgement (self-assessment) leading to invalid conclusions on decisions made.

The model described is underpinned by a research base.11^–13^,15^–23 However, further evidence from a range of GP groups as to its overall utility is necessary to inform future development.24^,25 GP appraisers are a key group with important professional insights into the strengths and weakness of the appraisal system and how it may be enhanced. Against this background, this study aimed to evaluate GP appraisers' views of the acceptability, feasibility, and educational impact of independent, external peer feedback received on one of three core appraisal activities they undertook as part of this study. In this way, GP appraisers would gain experience of receiving feedback on specific aspects of their own appraisal activities as part of the NES model described. Appraisers could then judge the usefulness of this approach as a support mechanism for the GP appraisal system in NHS Scotland.

METHOD

Completion of core appraisal tasks

Sample and recruitment

All 164 GP appraisers in NHS Scotland were randomly split into three groups, and one of three appraisal activities (SEA, criterion audit, or videotaped consultations) was allocated to each. To maximise fairness for participants and encourage engagement with the study, group numbers were adjusted to account for the effort needed to perform each activity (for example, fewer individuals were allocated to video consultation than SEA because the former is much more onerous). Appraisers were invited to undertake the allocated activity within a 4-month timeframe and submit this to the NES peer feedback model. Informed consent was obtained by providing a full written description and purpose of the evaluation. A standardised re-imbursement fee was offered by the host organisation (NES) to cover back-fill costs for GP participation time.

How this fits in

GP appraisal is considered inadequate and needs to be more robust and objective. The quality of core appraisal activities undertaken by GPs is also known to be variable. Independent feedback by trained peers on these activities is proposed as a potential educational solution. This study provides evidence that GP appraisers who received peer feedback on appraisal activities found it acceptable and educationally useful. Furthermore, they agreed it would ‘add value’ in improving the quality of appraisal evidence.

The model involves the application of formative assessment instruments,11^,15^,16 by trained GP peers to facilitate developmental feedback for participants. For audit and SEA submissions, two GP peers (from a group of 20) are allocated to independently review submitted reports, with a single reviewer used for videos of consultations (from a group of 20). A GP peer review coordinator then collates this written feedback on performance, provides their own input, and compiles a short report which is sent to participants.

Postal questionnaire survey

Data collection

A questionnaire was developed to examine attitudes towards the feasibility, acceptability, and educational impact of external peer feedback on the relevant appraisal activity undertaken. The questionnaire was pre-tested with four GP appraiser colleagues. The survey took place in March 2008, 3 days after peer feedback reports were sent to participants. Non-responders were emailed reminders until all questionnaires were returned. Levels of agreement with attitudinal statements were indicated on a 7-point rating scale.

Data analysis

Data were entered into a Microsoft Excel spreadsheet. The number of rating scores ≥4 was calculated. A Wilcoxon signed-rank test and 95% confidence interval (CI) for estimated median differences were performed to compare before and after rating scores, which were calculated using Minitab (version 15).

RESULTS

Participants and appraisal materials

A total of 80/164 GP appraisers agreed to participate (49%). Of the 80, 67 (84%) submitted appraisal materials for external peer review. All 67 participants returned completed questionnaires (100%).

Participants' reported apprehension and knowledge and skill levels before and after peer feedback

Table 2 outlines statistical differences in participants' reported levels of apprehension about submitting SEA (P = 0.52), audit (P = 0.14), and video (P = 0.04) for external peer feedback before and after the study. The differences in appraisers' estimated knowledge and skills levels before and after receiving feedback on SEA (P<0.01), audit (P = 0.06) and video consultations (P = 0.18) are also detailed in Table 2.

View this table:

Table 2

Reported levels of apprehension about submitting educational materials for external peer feedback and estimated knowledge and skill ratings (before and after the study).

Significant event analysis

A majority of potential participants allocated to the SEA group (91.2%; 44/48) submitted completed reports (Table 3). There was unanimous agreement (n = 44) that submitting one report every 5 years for external peer review would be feasible for most GPs. Responders believed the peer feedback received was fair (n = 42); would facilitate improvement in their SEA practice (n = 32); and would be acceptable to most colleagues (n = 42). The majority agreed (n = 42) that external feedback on SEA would add value to the existing appraisal system.

View this table:

Table 3

GP Appraisers' attitudes towards the feasibility, acceptability, and educational gain of peer-reviewed significant event analyses (n = 44).

Criterion audit

Fifteen of 18 expected criterion audit reports (83.3%) were received (Table 4). Most responders were unconcerned about their audit report being reviewed by external peers (n = 14) and believed this would also be acceptable to GP colleagues (n = 15). Feedback received provided specific advice on how to improve audit skills (n = 15) and would facilitate improvements in related practices for the majority (n = 12). A majority agreed (n = 14) that the existing appraisal system would be enhanced by external feedback on criterion audits.

View this table:

Table 4

GP appraisers' attitudes towards the feasibility, acceptability, and educational gain of peer-reviewed criterion audit (n = 15).

Videotaped consultations

A total of 8/14 appraisers (57.1%) submitted videos of consultations for review (Table 5). Five responders believed completing a consultation video was feasible as part of normal general practice. Peer feedback on consulting skills was perceived by all participant appraisers to be fair and educationally useful and to have the potential to have an impact on the consulting skills of most GP colleagues. There was a unanimous perception that peer-reviewed video consultations would add value to the appraisal process.

View this table:

Table 5

GP Appraisers' attitudes towards the feasibility, acceptability and educational gain of peer-reviewed video consultations (n = 8).

DISCUSSION

Summary of main findings

This study found strong support from participants for the feasibility, acceptability, and educational gain associated with the external and independent review of three core appraisal activities by trained colleagues. Many expressed a certain level of apprehension about receiving external feedback on their appraisal activities, but this decreased slightly after participation, particularly for those submitting a video of consultations. Additionally, there was a general perception that review of core appraisal activities by trained colleagues would ‘add value’ to the current appraisal system in Scotland.

Peer feedback received on SEA and criterion audit reports helped responders to understand more about these techniques, and engendered a reported commitment to practise it more effectively. There was a perceived increase in estimated knowledge and skills levels associated with each activity after peer feedback had been received, particularly for SEA. However, a large minority experienced low-to-moderate education gains from participation. This could be explained by the fact that most GP appraisers should be experienced in SEA and audit. Related evidence and knowledge are also required for many educational and governance reasons,1 including training practice accreditation and to benefit from the Quality and Outcomes Framework.25 Additional knowledge will also be gleaned by appraisers when reflecting on SEA and audit reports with appraisees. It should also be acknowledged that a proportion of GPs will be highly effective in applying both techniques, and that peer feedback, while reinforcing excellent performance in these cases, may offer minimal educational gain. Given this background, there is a strong perception from appraisers that other GP colleagues would benefit from developmental feedback by trained peers on SEA and audit attempts, which reflects previous research evidence.17^–20^,22

A comparatively small number of appraisers submitted consultation videos for peer review. Although there was strong agreement that other GPs would benefit from this method of receiving feedback on their consultations skills, there was less agreement on whether this would be acceptable to most GPs. Indeed previous work has demonstrated that the process can be effectively undertaken by a GP without previous experience in this area,21 however, peer review of video consultation is often perceived to be much more challenging and stressful than participating in audit or SEA.26 The process can often take longer and be technically difficult to organise and execute, while patient consent is also required. Perhaps most importantly it is also perceived to be more professionally threatening than the other two activities.26 However, there was strong positive agreement on the potential value of external peer review of video consultations educationally and when linked to the GP appraisal system.

Strengths and limitations of the study

A large minority of all Scottish appraisers participated. However, the study population was self-selecting and potentially biased as the views and experiences of non-participating appraisers and other GP groups may differ from those reported. The study context is specific to NHS Scotland, but the underlying principle linking external peer review and appraisal is potentially relevant to other medical specialties and healthcare systems. Most appraisers who agreed to participate did so, but a substantial minority failed to submit video consultations, which points to a potential difficulty in linking this activity with the peer review model as part of normal general practice. Written feedback was provided by peer reviewers. However, in some situations, face-to-face feedback might be more desirable but this also creates a feasibility issue.

Comparison with existing literature

Given the increasing importance being accorded to the links between appraisal and revalidation,8^–10 more attention will inevitably be paid to the standard of evidence being submitted. Ultimately, someone has to make a judgement as to whether it is good enough. Appraisers have consistently voiced their discomfort at being given this task.6^,7 This external peer review system allows the judgements to be made prior to an appraisal with a built-in sampling process of ‘quality control’ ensuring fairness across the country, that is, a national standard. However, the feasibility of a national external peer review system to support appraisal is open to question. For such a system to function, additional resources will be required. One option is to consider the vision that all GPs are potential peer reviewers — with appropriate training — with local ‘experts’ taking on the role of quality assuring the process. If it is to be practicable, much wider discussion is needed. The status quo, however, may not be an option.

The literature on the limited value of self-assessment is clear.27 The benefit, therefore, of providing additional peer review should be not only desirable but — given the stakes — essential. The adaptation of the Hammond model of Cognitive Continuum Theory underpins this justification by encouraging a more rational (rather than merely intuitive) approach to decision making on material submitted for appraisal and inclusion in a revalidation folder.

A call for external verification of evidence will not be straightforward — GP appraisers are in the frontline and, in their opinion in this study, those who took part felt the advantages outweighed the disadvantages by suggesting it gave added value to the appraisal process. In their proposal to improve the appraisal process, the Royal College of General Practitioners suggests that over a 5-year period GPs should present evidence of a specific number of SEA and completed audit cycle reports for discussion during appraisals.28 This represents an increase in the quantity of evidence that is currently required. However, it is unclear how this overcomes the perceived inadequacy of the present approach, which is that the appraisal system lacks a robust method to verify the quality of evidence that is already known to be variable.17^–20^,22

Implications for future research

Future research on this model should focus on shedding light on the acceptability and feasibility of peer reviewing videos of consultations on a large-scale basis. More confirmatory work on strengthening the inter-rater reliability of video consultation peer review is also required. Overall, there is a need to explore with decision makers if and how independent peer review can make a meaningful contribution to improving the robustness of evidence required for appraisal in support of revalidation.

Acknowledgments

We wish to thank all GP appraisers and GP peer reviewers in NHS Scotland who participated. The following individuals helped to coordinate the study and we extend our grateful thanks to them, also: Marjorie McArthur, Kirsten Macdonald, Dr Ed Russell-Smith and Dr Gordon McLeay. We are also grateful to Dr Lilian Murray, University of Glasgow, for statistical advice.

Notes

Funding body

This study was funded by NHS Education for Scotland

Ethic approval

The study was judged by the authors to be service evaluation

Competing interests

The authors' employing organisation is responsible for managing the national GP appraisal scheme and also manages the peer review model outlined in the study. The views expressed reflect those of the authors and not NES

Received February 26, 2009.
Revision received April 17, 2009.
Accepted May 18, 2009.

REFERENCES

↵
1. Scottish Executive, NHS Education for Scotland, RCGP (Scotland) & BMA (Scotland)
(2003) GP appraisal scheme: a brief guide (Scottish Executive, Edinburgh) http://www.sehd.scot.nhs.uk/publications/gpabg/gpabg-00.htm (accessed 2 Jun 2009).
↵
1. General Medical Council
(2006) Good medical practice (GMC, London) http://www.gmc-uk.org/guidance/good_medical_practice/GMC_GMP.pdf (accessed 30 May 2009).
↵
1. McKinstry B,
2. Shaw J,
3. McGillvray L,
4. Skinner L
(2002) What do general practitioners think about annual appraisal? A questionnaire-based cross-sectional study in southeast Scotland. Educ Prim Care 13(4):472–476.
OpenUrl
1. Jelley D,
2. van Zwnanenberg T
(2001) Peer appraisal in general practice: descriptive study in the Northern Deanery. Educ Gen Pract 11(1):281–287.
OpenUrl
↵
1. Lewis M,
2. Welwyn G,
3. Wood F
(2003) Appraisal of family doctors: an evaluation study. Br J Gen Pract 53(491):454–460.
OpenUrl Abstract/FREE Full Text
↵
1. Coltart I,
2. Cameron N,
3. McKinstry B,
4. Blaney D
(2008) What do doctors really think about the relevance and impact of GP appraisal 3 years on? A survey of Scottish GPs. Br J Gen Pract 58(547):82–87.
OpenUrl Abstract/FREE Full Text
↵
1. Wakeling J,
2. Cameron N,
3. Lough M
(2008) What factors impact on general practitioners' engagement with appraisal? A qualitative evaluation in Scotland. Educ Prim Care 19(6):615–623.
OpenUrl
↵
1. General Medical Council
(2003) License to practise and revalidation for doctors (GMC, London) http://www.gmc-uk.org/doctors/documents/archive/licence_to_practise.pdf (accessed 2 Jun 2009).
↵
1. Secretary of State for Health
(2007) Trust, assurance and safety — the regulation of health professionals in the 21st century. White Paper presented to Parliament by the Secretary of State for Health by Command of Her Majesty (the Stationery Office, London) http://www.gmc-uk.org/doctors/documents/archive/licence_to_practise.pdf (accessed 2 Jun 2009).
↵
1. Department of Health
(2006) Good doctors, safer patients: proposals to strengthen the system to assure and improve the performance of doctors and to protect the safety of patients. A report by the Chief Medical Officer (Department of Health, London) http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4137232 (accessed 2 Jun 2009).
↵
1. Cameron N,
2. McMillan R
(2006) Enhancing communication skills by peer review of consultation videos. Educ Prim Care 17(1):40–48.
OpenUrl
1. Bowie P,
2. McCoy S,
3. McKay J,
4. Lough M
(2005) Learning issues raised by the educational peer review of significant event analyses in general practice. Qual Prim Care 13(2):75–84.
OpenUrl
↵
1. McKay J,
2. Bowie P,
3. Lough M
(2006) Variations in the ability of general medical practitioners in applying two methods of clinical audit: a five-year study of assessment by peer review. J Eval Clin Pract 12(6):622–629.
OpenUrl CrossRef PubMed
↵
1. Hammond KR
1. Hammond KR
(1978) in Judgement and decision in public policy formation, Towards increasing competence of thought in public policy formation, ed Hammond KR (Westview Press, Boulder, CO), pp 11–32.
↵
1. Lough JRM,
2. McKay J,
3. Murray TS
(1995) Audit and summative assessment: a criterion-referenced marking schedule. Br J Gen Pract 45(400):607–609.
OpenUrl Abstract/FREE Full Text
↵
1. McKay J,
2. Murphy D,
3. Bowie P,
4. et al.
(2007) Development and testing of an assessment instrument for the formative peer review of significant event analyses. Qual Saf Health Care 16(2):150–153.
OpenUrl Abstract/FREE Full Text
↵
1. Bowie P,
2. McKay J,
3. Dalgetty E,
4. Lough M
(2005) A qualitative study of why general practitioners may participate in significant event analysis and educational peer review. Qual Saf Health Care 14(3):185–189.
OpenUrl Abstract/FREE Full Text
1. McKay J,
2. Shepherd A,
3. Bowie P,
4. Lough M
(2008) Acceptability and educational impact of a peer feedback model for significant event analysis. Med Educ 42(12):1210–1217.
OpenUrl CrossRef PubMed
1. McKay J,
2. Pope L,
3. Bowie P,
4. Lough M
(2009) Peer feedback in general practice: a focus group study of trained reviewers of significant event analyses. J Eval Clin Pract 15(1):142–147.
OpenUrl CrossRef PubMed
↵
1. Murie J,
2. McCrae J,
3. Bowie P
(2009) The peer review pilot project: a potential system to support GP appraisal in NHS Scotland. Educ Prim Care 20:34–40.
OpenUrl PubMed
↵
1. McMillan R,
2. Cameron N
(2006) Factors influencing the submission of videotaped consultations by general practitioners for peer review and educational feedback. Qual Prim Care 14(2):85–89.
OpenUrl
↵
1. Bowie P,
2. Cooke S,
3. Lo P,
4. et al.
(2007) The assessment of criterion audit cycles by external peer review — when is an audit not an audit? J Eval Clin Pract 13(3):352–357.
OpenUrl CrossRef PubMed
↵
1. Van der Vleuten CPM
(1996) The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ Theory Pract 1(1):41–67.
OpenUrl CrossRef PubMed
↵
1. Evans R,
2. Elwyn G,
3. Edwards A
(2004) Review of instruments for peer assessment of physicians. BMJ 328(7458):1240–1243.
OpenUrl Abstract/FREE Full Text
↵
1. Department of Health
(2006) New GMS contract 2006/7 (Stationery Office, London) http://www.dh.gov.uk/en/Healthcare/Primarycare/Primarycarecontracting/GMS/DH_4125636 (accessed 2 Jun 2009).
↵
1. NHS Modernisation Agency
(2004) Defining the evidence for revalidation — supporting the Royal College of General Practitioners. Collation of views from the NHS Clinical Governance Support Team Expert Group (NHS), http://www.gp-training.net/cme/revalidation/revalidation_evidence.pdf (accessed 2 Jun 2009).
↵
1. Coltart A,
2. Bagnall G,
3. Evans A,
4. et al.
Best evidence in medical education (BEME). A systematic review of the literature on the effectiveness of self-assessment in clinical education, http://www.bemecollaboration.org/beme/pages/reviews/mckinstry.html (accessed 27 May 2009).
↵
1. Royal College of General Practitioners
(2008) RCGP Guide to the Revalidation of general practitioners (Royal College of General Practitioners, London) http://www.rcgp.org.uk/PDF/PDS_Guide_to_Revalidation_for_GPs_April_2009_V1.0.pdf (accessed 30 May 2009).

In this issue

Download PDF

Email Article

Citation Tools

Keywords

More in this TOC Section

Show more Original Papers

Cited By...

Intended for Healthcare Professionals

[1] ↵
Scottish Executive, NHS Education for Scotland, RCGP (Scotland) & BMA (Scotland)
(2003) GP appraisal scheme: a brief guide (Scottish Executive, Edinburgh) http://www.sehd.scot.nhs.uk/publications/gpabg/gpabg-00.htm (accessed 2 Jun 2009).

[2] Scottish Executive, NHS Education for Scotland, RCGP (Scotland) & BMA (Scotland)

[3] ↵
General Medical Council
(2006) Good medical practice (GMC, London) http://www.gmc-uk.org/guidance/good_medical_practice/GMC_GMP.pdf (accessed 30 May 2009).

[4] General Medical Council

[5] ↵
McKinstry B,
Shaw J,
McGillvray L,
Skinner L
(2002) What do general practitioners think about annual appraisal? A questionnaire-based cross-sectional study in southeast Scotland. Educ Prim Care 13(4):472–476.
OpenUrl

[6] McKinstry B,

[7] Shaw J,

[8] McGillvray L,

[9] Skinner L

[10] Jelley D,
van Zwnanenberg T
(2001) Peer appraisal in general practice: descriptive study in the Northern Deanery. Educ Gen Pract 11(1):281–287.
OpenUrl

[11] Jelley D,

[12] van Zwnanenberg T

[13] ↵
Lewis M,
Welwyn G,
Wood F
(2003) Appraisal of family doctors: an evaluation study. Br J Gen Pract 53(491):454–460.
OpenUrl Abstract/FREE Full Text

[14] Lewis M,

[15] Welwyn G,

[16] Wood F

[17] ↵
Coltart I,
Cameron N,
McKinstry B,
Blaney D
(2008) What do doctors really think about the relevance and impact of GP appraisal 3 years on? A survey of Scottish GPs. Br J Gen Pract 58(547):82–87.
OpenUrl Abstract/FREE Full Text

[18] Coltart I,

[19] Cameron N,

[20] McKinstry B,

[21] Blaney D

[22] ↵
Wakeling J,
Cameron N,
Lough M
(2008) What factors impact on general practitioners' engagement with appraisal? A qualitative evaluation in Scotland. Educ Prim Care 19(6):615–623.
OpenUrl

[23] Wakeling J,

[24] Cameron N,

[25] Lough M

[26] ↵
General Medical Council
(2003) License to practise and revalidation for doctors (GMC, London) http://www.gmc-uk.org/doctors/documents/archive/licence_to_practise.pdf (accessed 2 Jun 2009).

[27] General Medical Council

[28] ↵
Secretary of State for Health
(2007) Trust, assurance and safety — the regulation of health professionals in the 21st century. White Paper presented to Parliament by the Secretary of State for Health by Command of Her Majesty (the Stationery Office, London) http://www.gmc-uk.org/doctors/documents/archive/licence_to_practise.pdf (accessed 2 Jun 2009).

[29] Secretary of State for Health

[30] ↵
Department of Health
(2006) Good doctors, safer patients: proposals to strengthen the system to assure and improve the performance of doctors and to protect the safety of patients. A report by the Chief Medical Officer (Department of Health, London) http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4137232 (accessed 2 Jun 2009).

[31] Department of Health

[32] ↵
Cameron N,
McMillan R
(2006) Enhancing communication skills by peer review of consultation videos. Educ Prim Care 17(1):40–48.
OpenUrl

[33] Cameron N,

[34] McMillan R

[35] Bowie P,
McCoy S,
McKay J,
Lough M
(2005) Learning issues raised by the educational peer review of significant event analyses in general practice. Qual Prim Care 13(2):75–84.
OpenUrl

[36] Bowie P,

[37] McCoy S,

[38] McKay J,

[39] Lough M

[40] ↵
McKay J,
Bowie P,
Lough M
(2006) Variations in the ability of general medical practitioners in applying two methods of clinical audit: a five-year study of assessment by peer review. J Eval Clin Pract 12(6):622–629.
OpenUrl CrossRef PubMed

[41] McKay J,

[42] Bowie P,

[43] Lough M

[44] ↵
Hammond KR
Hammond KR
(1978) in Judgement and decision in public policy formation, Towards increasing competence of thought in public policy formation, ed Hammond KR (Westview Press, Boulder, CO), pp 11–32.

[45] Hammond KR

[46] Hammond KR

[47] ↵
Lough JRM,
McKay J,
Murray TS
(1995) Audit and summative assessment: a criterion-referenced marking schedule. Br J Gen Pract 45(400):607–609.
OpenUrl Abstract/FREE Full Text

[48] Lough JRM,

[49] McKay J,

[50] Murray TS

[51] ↵
McKay J,
Murphy D,
Bowie P,
et al.
(2007) Development and testing of an assessment instrument for the formative peer review of significant event analyses. Qual Saf Health Care 16(2):150–153.
OpenUrl Abstract/FREE Full Text

[52] McKay J,

[53] Murphy D,

[54] Bowie P,

[55] et al.

[56] ↵
Bowie P,
McKay J,
Dalgetty E,
Lough M
(2005) A qualitative study of why general practitioners may participate in significant event analysis and educational peer review. Qual Saf Health Care 14(3):185–189.
OpenUrl Abstract/FREE Full Text

[57] Bowie P,

[58] McKay J,

[59] Dalgetty E,

[60] Lough M

[61] McKay J,
Shepherd A,
Bowie P,
Lough M
(2008) Acceptability and educational impact of a peer feedback model for significant event analysis. Med Educ 42(12):1210–1217.
OpenUrl CrossRef PubMed

[62] McKay J,

[63] Shepherd A,

[64] Bowie P,

[65] Lough M

[66] McKay J,
Pope L,
Bowie P,
Lough M
(2009) Peer feedback in general practice: a focus group study of trained reviewers of significant event analyses. J Eval Clin Pract 15(1):142–147.
OpenUrl CrossRef PubMed

[67] McKay J,

[68] Pope L,

[69] Bowie P,

[70] Lough M

[71] ↵
Murie J,
McCrae J,
Bowie P
(2009) The peer review pilot project: a potential system to support GP appraisal in NHS Scotland. Educ Prim Care 20:34–40.
OpenUrl PubMed

[72] Murie J,

[73] McCrae J,

[74] Bowie P

[75] ↵
McMillan R,
Cameron N
(2006) Factors influencing the submission of videotaped consultations by general practitioners for peer review and educational feedback. Qual Prim Care 14(2):85–89.
OpenUrl

[76] McMillan R,

[77] Cameron N

[78] ↵
Bowie P,
Cooke S,
Lo P,
et al.
(2007) The assessment of criterion audit cycles by external peer review — when is an audit not an audit? J Eval Clin Pract 13(3):352–357.
OpenUrl CrossRef PubMed

[79] Bowie P,

[80] Cooke S,

[81] Lo P,

[82] et al.

[83] ↵
Van der Vleuten CPM
(1996) The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ Theory Pract 1(1):41–67.
OpenUrl CrossRef PubMed

[84] Van der Vleuten CPM

[85] ↵
Evans R,
Elwyn G,
Edwards A
(2004) Review of instruments for peer assessment of physicians. BMJ 328(7458):1240–1243.
OpenUrl Abstract/FREE Full Text

[86] Evans R,

[87] Elwyn G,

[88] Edwards A

[89] ↵
Department of Health
(2006) New GMS contract 2006/7 (Stationery Office, London) http://www.dh.gov.uk/en/Healthcare/Primarycare/Primarycarecontracting/GMS/DH_4125636 (accessed 2 Jun 2009).

[90] Department of Health

[91] ↵
NHS Modernisation Agency
(2004) Defining the evidence for revalidation — supporting the Royal College of General Practitioners. Collation of views from the NHS Clinical Governance Support Team Expert Group (NHS), http://www.gp-training.net/cme/revalidation/revalidation_evidence.pdf (accessed 2 Jun 2009).

[92] NHS Modernisation Agency

[93] ↵
Coltart A,
Bagnall G,
Evans A,
et al.
Best evidence in medical education (BEME). A systematic review of the literature on the effectiveness of self-assessment in clinical education, http://www.bemecollaboration.org/beme/pages/reviews/mckinstry.html (accessed 27 May 2009).

[94] Coltart A,

[95] Bagnall G,

[96] Evans A,

[97] et al.

[98] ↵
Royal College of General Practitioners
(2008) RCGP Guide to the Revalidation of general practitioners (Royal College of General Practitioners, London) http://www.rcgp.org.uk/PDF/PDS_Guide_to_Revalidation_for_GPs_April_2009_V1.0.pdf (accessed 30 May 2009).

[99] Royal College of General Practitioners

Main menu

User menu

Search

Verifying appraisal evidence using feedback from trained peers: views and experiences of Scottish GP appraisers

Abstract

INTRODUCTION

METHOD

Completion of core appraisal tasks

Sample and recruitment

How this fits in

Postal questionnaire survey

Data collection

Data analysis

RESULTS

Participants and appraisal materials

Participants' reported apprehension and knowledge and skill levels before and after peer feedback

Significant event analysis

Criterion audit

Videotaped consultations

DISCUSSION

Summary of main findings

Strengths and limitations of the study

Comparison with existing literature

Implications for future research

Acknowledgments

Notes

Funding body

Ethic approval

Competing interests

REFERENCES

In this issue

Citation Manager Formats

Jump to section

Keywords

More in this TOC Section

Related Articles

Cited By...

NAVIGATE

RCGP

MY ACCOUNT

NEWS AND UPDATES

AUTHORS & REVIEWERS

CUSTOMER SERVICES

CONTRIBUTE

CONTACT US