Abstract
Background Patient safety is a key issue in primary care. Significant event analysis (SEA) is a long established method of improving safety. In 2004, SEA was introduced as part of the Quality and Outcomes Framework (QOF) of the new general medical services (GMS) contract.
Aim To review SEAs submitted for the QOF by general practices for a primary care trust (PCT) in 2004–2005.
Design of study A retrospective review of SEAs.
Setting St Helens PCT, Merseyside, North West England, UK (185 000 patients), now part of Halton and St Helens PCT.
Method Three hundred and thirty-seven QOF-reported SEAs were reviewed from 32 (91 %) of a total of 35 St Helens PCT practices (mean 10.5, range 4–17).
Results Practices identified learning points in 89% of SEAs. Twenty-two of 32 (69%) practices successfully performed SEA and required no further support. Four practices identified learning points but needed further facilitation in implementing change or actions arising from SEA. Six practices had significant difficulties with SEA processes and were referred for extra SEA training locally. Ninety (26.7%) of all significant events were classified as patient-safety incidents. Of these, 22 (6.5%) were ‘serious or life threatening’ and 67 (19.9%) were ‘potentially serious’. Ninety-six (28.5%) of the significant events related to medicines management issues; and 63 (18.7%) had key learning points for partnership organisations. Main outcome measures were review of SEA process as a team learning event; QOF significant event criteria; National Patient Safety Agency classification of significant events, and category of patient-safety incidents.
Conclusion SEA in general practice is a valuable clinical governance and educational tool with potential patient safety benefits. Most practices performed SEA successfully but there were performance concerns and patient-safety issues were highlighted. This review emphasises the need for primary care organisations to be able to analyse and share SEAs effectively.
INTRODUCTION
Significant event analysis (SEA) attempts to bring orderly review and effective action to the complex systems of health care. It has been defined as occurring when:
‘…individual cases in which there has been a significant occurrence (not necessarily involving an undesirable outcome for the patient), are analysed in a systematic and detailed way to ascertain what can be learnt about the overall quality of care and to indicate changes that might lead to future improvements’.1
SEA was first described when the number of medical audits increased dramatically with the 1990 NHS reforms,2 but goes back at least 30 years to the first analyses of deaths in general practice.3
Most GPs have their first experience of SEA in problem case-analysis tutorials as registrars. However, It has, been estimated that only a small proportion, perhaps 6%, of all adverse events in the NHS are reported,4 and little has been published about SEA in general practice.5
Formal SEA was a minority activity until the 2004 Quality and Outcomes Framework (QOF) rewarded practices for carrying out either six or 12 significant events audits in the previous 3 years.6 The new general medical services (GMS) contract appears to have delivered the first near-universal reporting of SEAs by NHS practices to PCTs.
However, when any activity changes from being the preserve of innovators and early adopters to majority participation, its nature and effects risk being changed.7 The dual aims of this retrospective review were to learn more about the process of QOF SEA reporting within one district, and to understand more about the types of significant events that practices were prepared to report within this annual process. This will enable an understanding of how SEA in general practice has withstood the transition from minority to mainstream activity.
St Helens PCT was responsible for 35 general practices caring for 185 000 patients in an industrial town in North West England. Since 2000, several educational events have been held each year to promote and explain SEA to these practices.
QOF data and visit reports are public documents subject to the Freedom of Information Act. St Helens' practices have a long history of sharing data, such as prescribing and referrals, in an open, identifiable format. SEAs and their processes were discussed at the QOF visits. This review was further discussed with, and approved by, the PCT chief executive and chair, and the Governance Team.
METHOD
All of the GP significant events submitted to the St Helens PCT for the year 2004–2005 were analysed. Events were assessed by practice according to process analysis and degree of seriousness.
Process analysis
Practice SEA case reports were compared with a standardised SEA format,1 which advocated the identification of learning points; an action plan with implementation dates and shared learning where appropriate; and a resultant review or audit. This process has been a constant feature of the SEA training locally.
How this fits in
Reporting significant event analysis (SEA) was an infrequently performed activity in UK general practice until financially rewarded by the 2004 Quality and Outcomes Framework. The effects on the SEA process of this transition to mainstream activity were unknown. Reports in one district suggested that most practices could complete the process of SEA appropriately. However, a substantial minority struggled to identify and implement the actions required to improve patient safety.
Degree of seriousness
Events were also classified according to established National Patient Safety Agency (NPSA) and QOF criteria (Boxes 1 and 2) by type and seriousness. Events that were considered to be ‘patient-safety incidents’ were categorised according to an abbreviated version of the NPSA classification:8 serious or life-threatening (‘red’); potentially serious (‘amber’); or unlikely to be life threatening (‘green’).
Box 1. Classification of types of significant events developed from expanded QOF scheme.
▸ Death occurring in practice premises
▸ New cancer diagnoses
▸ Death where terminal care took place at home
▸ Patient complaint
▸ Suicide
▸ Section under Mental Health Act
▸ Other administrative
▸ Prescribing-related events
▸ Nursing-related events
▸ Other medical
▸ Other
QOF = Quality and Outcomes Framework.
Box 2. Seriousness and necessary action based on NPSA scheme.
Degree of seriousness
Red (serious or life threatening)
Amber (potentially serious)
Green (unlikely to be life threatening)
Necessary action
Planned and implemented
Planned
Not identified
Not applicable
NPSA = National Patient Safety Agency.
The PCT governance team discussed events where the classification was in doubt, or which were considered to be serious or life threatening. Reviewers shared the issues raised with the practices and a consensus on necessary action was reached.
RESULTS
Thirty-three (94.3%) of the 35 practices submitted SEAs for the QOF process. One practice chose to retain their submitted data, which meant these data were not available to be analysed.
Process analysis
Practices produced learning points from each SEA in 300 cases (89.0%), and action plans in 248 cases (73.6%). At the time of submission for their QOF (31 March 2005), practices reported implemented action plans in 147 SEA's (43.6%). Table 1 summarises the practice processes for SEA and the number of serious or life-threatening events.
Table 1 Practices' analyses of their significant events, and distribution of serious and/or life-threatening events.
Sharing SEA findings internally within teams was recorded in 262/337 events (77.7%). Sixty-three out of 337 events (18.7%) originated outside practices and within partnership organisations, of which 42 (66.7%) were shared by practices with that organisation.
Degree of seriousness
A total of 337 SEAs from 32 practices (mean 10.5 per practice; range 4–17) were analysed. In varying degrees of seriousness, there were 90 events (26.7%) that were patient-safety incidents according to NPSA classification (range 0–9 per practice). Of these, 22 (6.5%) were ‘red’ significant events (serious or life threatening; range 0–3 per practice) and 67 (19.9%) were ‘amber’ significant events (potentially serious; range 2–13 per practice). The one remaining significant event was assessed as ‘green’ (unlikely to be life threatening).
Categories of the significant events reported are shown in Table 2, with serious or life-threatening SEAs in summarised format. Prescribing events and other medicines management events (part of ‘other administrative’) accounted for 96 (28.5%) of the total number of SEAs.
Table 2 Summary of significant events.
DISCUSSION
Summary of main findings
Self-analyses of the SEA process by 22 (69%) practices showed that, irrespective of size, practices were performing SEA in a systematic fashion requiring no further support. Four practices appeared to require additional facilitation to implement their identified action plans, but six appeared to have significant difficulties with the whole process of SEA. Table 1 shows that these six practices struggling with the SEA process were small practices (≤2 partners in GP practices).
SEA has been considered an effective educational strategy for general practice.9 Its strength is probably linked to adult learning theory10 and doctors' predilection for tell anecdotes.11 The authors were among many who had facilitated SEA;1,12,13 over 10 years finding that most participants could understand and plan the process in facilitated groups. This study also revealed that most practice teams were able to review SEAs and make plans to prevent their recurrence. However, there remains a need to help practices with the implementation of actions identified in SEA, as the action required to improve practice is often more difficult than initially realised.14
SEA is a ‘dynamic technology’, developing rather than being fixed, and inevitably being changed by local practitioners as they adopt it.15 The regulations specify that some events (such as suicide, or death on practice premises) be included, but are otherwise open to broad interpretation. The right set of rewards seems to have been devised for practices to analyse SEAs, but they do not automatically result in effective change.
Strengths and limitations of the study
These routinely reported SEAs are the largest published single series that the authors have discovered from general practice in the UK. Their collection from the routine work of practices across a district should reduce the selection bias inherent in many special projects which commonly involve enthusiasts alone.
The investigation of a large series of SEA reports depends on the understanding of process within practices and their willingness to report appropriate incidents; the format of the reporting scheme; and subsequent analyses. Although practices could have reported incidents in a selective manner, the wide spectrum of events reported here along with the training events observed, suggests this is an appropriate reflection of general practice.
Furthermore, Table 2 shows that 22 serious or life-threatening events were reported, implying local arrangements were trusted to some extent at least. Although it is impossible to know if some events were culled either before or after open discussion in practices, there appears to be openness in the sharing of significant events locally which, it may be assumed, the QOF process has contributed to.
The selection from a single district may limit the application to other settings, perhaps those with a very different history or culture.
Comparison with existing literature
Mechanisms for learning from adverse events in primary care are considered to be less well developed than those in hospitals.16 Little work has been carried out into the analysis of patient safety in primary care.17 Reflection and action from SEAs should be a major part of the drive for quality improvements in the NHS.18 In an analysis of the process of 662 events occurring in Scottish general practices in 2000–2004, most ‘unsatisfactory’ SEAs also showed problems with implementing appropriate action plans.5 Prescribing problems were also a major event category in that series, emphasising the importance of medicines management for patient safety.
An influential Department of Health report, An Organisation with a Memory,16 encouraged the development of an ‘informed organisational culture’ where errors and near-misses could be assessed as the basis for developing safer organisations. The QOF for general practice has now encouraged and rewarded this.
Implications for clinical practice and future research
From public PCT records, 33/35 (94.3%) practices qualified for QOF payments by submitting SEAs, usually by submitting the full quota of 12 SEAs (28/35, 80%). SEAs cost the NHS in England about £4 million in direct payments in 1 year, excluding SEA training and facilitation costs. The events in Table 2 show that a proper system for SEA could deliver savings from greater patient safety and preventable litigation far exceeding this cost.
This review discovered 63/337 (18.7%) SEAs where the major learning points were with another organisation, such as the local hospital. In 42 cases (67%) these learning points were shared appropriately; but in 21 (33%) sharing did not take place (6.2% of all SEAs). This remains a potential weakness of current schemes.
Conclusions and implications for the NHS
In these general-practice SEAs, clear learning points were identified in most (89%) cases. In many cases (74%), action plans had been produced or implemented by the time of submission for the QOF.
SEAs within the QOF reveal many events with important patient safety learning benefits. If SEA is effectively facilitated it is expected to be both cost-effective and increasingly used. The development of local SEA systems, and shared learning with partnership organisations, could do much to support this. Struggling general practice teams are usually ready for help with the SEA process, and this is a critical opportunity to improve patient safety. Primary care organisations and national agencies should build on the momentum of the QOF and nurture the potential of this audit process.
Acknowledgments
We wish to thank our local GP colleagues, their teams and St Helens PCT.
Notes
Ethics committee
This is a retrospective review of documents held in the public domain by St Helens PCT
Funding body
Not applicable
Competing interests
The authors have stated that there are none
- Received January 5, 2006.
- Revision received October 16, 2006.
- Accepted May 8, 2007.
- © British Journal of General Practice, 2007.
