Abstract
Background Centralised incident reporting in a Dutch collaboration of nine out-of-hours services yielded very few incident reports. To improve incident reporting and the awareness of primary caregivers about patient safety issues, a local incident-reporting procedure was implemented.
Aim To compare the number and nature of incident reports collected in a local incident-reporting procedure (intervention) versus the currently used centralised incident-reporting procedure.
Design of study Quasi experiment.
Setting Three GPs' out-of-hours services (OHSs) in the centre of the Netherlands participated over 2 years before and 2 years after the intervention.
Method A local incident-reporting procedure was implemented in OHS1, in which participants were encouraged to report all occurring incidents. A local committee with peers analysed the reported incidents fortnightly in order to initiate improvements if necessary. In OHS2 and OHS3, the current centralised incident-reporting procedure was continued, where incidents were reported to an advisory committee of the board of directors of the OHSs collaboration and were assessed every 2 months. The main outcome measures were the number and nature of incidents reported.
Results At baseline, participants reported fewer than 10 incidents per year each. In the follow-up period, the number of incidents reported in OHS1 increased 16-fold compared with the controls. The type of incidents reported did not alter. In the local incident-reporting procedure, improvements were implemented in a shorter time frame, but reports in the centralised incident-reporting procedure led to a more systematic addressing of general and recurring safety problems.
Conclusion It is likely that a local incident-reporting procedure increases the willingness to report and facilitates faster implementation of improvements. In contrast, the central procedure, by collating reports from many settings, seems better at addressing generic and recurring safety issues. The advantages of both approaches should be combined.
INTRODUCTION
Patient safety has become an important issue in hospital care, and awareness about safety is rapidly growing in primary care.1,2 One of the tools to increase patient safety is incident reporting and analysis.3,4 However, incident reporting in primary care is still relatively uncommon, as judged from the low number of reports from GPs to centralised reporting systems.5–8
Changes in Dutch primary care, moving from small practices and informal local groups of practices to larger professional organisations, most markedly concerning the organisation of out-of-hours services, has demonstrated the need for a structural incident reporting.9,10 Consequently, in 2005 the collaboration of nine GPs' out-of-hours services (OHSs) in the centre of the Netherlands initiated an incident-reporting procedure in which every incident was evaluated by an advisory committee of the board of directors of the OHS collaboration. However, in the first 2 years of the programme, very few incidents were reported.
Therefore, to improve incident reporting and the awareness of primary caregivers about patient-safety issues, a local incident-reporting procedure (LIRP) was designed, based on publications suggesting that ‘local’, meaning practice- or unit-based, incident reporting and analysis yields more incident reports than reporting procedures that are organised outside the actual workplace.11–13 In an earlier study it was shown that a LIRP is feasible in daycare general practice.14
The present study aimed to determine whether implementation of a LIRP in a collaborative GP's OHS changes the number and nature of incident reports, as well as any resulting improvement measures when compared to the current centralised incident-reporting procedure (CIRP).
METHOD
Design
The study was a non-equivalent controlled, quasi-experimental field study.15 Baseline incident-reporting data were collected at the end of 2006 relating to the period before the intervention (2005 and 2006). At the end of December 2006, the LIRP was implemented in OHS1. In OHS2 and OHS3, which served as controls, the current practice of reporting incidents to the central committee (CIRP) was continued. Follow-up data were obtained at the end of 2008 on all incidents reported in 2007 and 2008.
Setting
GPs' out-of-hours services in the Netherlands provide acute primary care in the evenings, nights, and weekends.10 Features of the participating OHSs are described in Table 1 (numbers from 2008).
Table 1 Features of the participating OHS locations.
Outcome measures
The outcome measures were the number of incident reports, type of incidents reported, and type of reporters, and an estimation of potential harm for the patient for each reported incident. Furthermore, qualitative data about any improvement measures initiated centrally or locally were obtained.
How this fits in
Primary caregivers scarcely report patient safety incidents in current regional or national databases. Reporting in the local setting seems to increase the willingness to report and facilitates faster implementation of improvements. In contrast, the central procedure, by collating reports from many settings, seems better at addressing generic and recurring safety issues. The advantages of both approaches should be combined.
Intervention
A LIRP was implemented in OHS1. The LIRP was based on a ‘plan-do-act-check’ learning cycle,16 similar to the LIRP in the ‘SPIEGEL’ study.14 All caregivers at the OHS were asked to report all incidents. An incident was defined as any unintended or unexpected event that could have led or did lead to harm for one or more patients receiving care.8 On paper forms, which were put into an ‘incident mail box’ on the OHS, the reporters were asked to write a narrative about what happened and to indicate the date, time, and place of the incident, who was involved, and whether there was any harm to the patient. A local, multidisciplinary incident-reporting procedure committee was trained to screen and analyse the incident reports. Incidents were selected for analysis by first assigning a risk score (0 to 4), based on an estimate of potential harm and the frequency of occurrence. The committee was advised to analyse incidents with a risk score of 2 or higher, based on PRISMA (Prevention and Recovery Information System for Monitoring and Analysis)17 and root cause analysis18 techniques. They were also responsible for feedback to reporters and to the organisation, and for development of improvement measures when appropriate.
In OHS2 and 3, the usual procedure and routine around incident reporting, CIRP, was continued. In the CIRP, incident reports were sent by mail to an advisory committee of the board of directors of the GP OHSs collaboration.
In the LIRP, the incident reports were assessed by physicians, medical nurses, and a chauffeur, who actually worked on that particular OHS location, instead of professionals who did not work in the OHS in which the incident occurred. In addition, the incident reports were processed in a much shorter feedback loop: 2–3 weeks, instead of 8–12 weeks for the CIRP.
In the first year after the introduction of LIRP, the research team supplied quarterly anonymised feedback information from the incident reports to the LIRP committee and, when requested, advised concerning current incident analyses. At the end of 2007 it was decided by the OHS management to continue LIRP. From that time on, LIRP was executed without the support of the study team.
Data collection
All incident reports of the LIRP and CIRP were anonymised and gathered in a research database. The research team categorised the incidents, using a classification derived from the literature,19,20 adapted for practical use in the Dutch situation.14 Potential harm was independently classified using an ORCE (Observe, Record, Classify, Evaluate) procedure,21 by two members of the research team. Differences were solved by discussion.
Data about the implementation of improvement measures were collected at the end of the research period by studying the year reports of each OHS and of the CIRP and LIRP committees. The location managers were asked to check and confirm the implementation of the proposed improvement measures. Furthermore, one researcher conducted open interviews with the location managers and members of the CIRP and the LIRP committees about their opinions on central and local incident reporting procedures, and concerning implementation barriers of the improvement measures.
Data analysis
Quantitative data were analysed with SPSS (version 15). For comparing the ‘before’ and ‘after’ data, χ2 or Fisher exact tests were used as appropriate. The data on the implementation of improvement were qualitatively assessed in the study team, by constant comparison.22
RESULTS
The number and nature of incident reports are shown in Figure 1.
Figure 1 Number and type of incident reports.
In 2005 and 2006, 10 (4 and 6 respectively) incidents were reported in OHS1, 14 (9 and 5) in OHS 2, and 17 (8 and 9) in OHS 3. After the intervention, 162 (126 and 36) incidents were reported in OHS1, and in the control OHSs 19 (11 and 8) were reported in OHS2 and 20 (13 and 7) in OHS3. In OHS 1, this meant a 25-fold increase in the number of incident reports compared to baseline in 2007 (P<0.001) and a sevenfold increase in 2008 (P = 0.004). The number of incident reports in the control OHS locations did not change over the study period.
In 2005 and 2006, the type of incidents reported in OHS1 were categorised as: process of care (five reports), knowledge and skills (two reports), materials and logistics (two reports), and communications/teamwork (one report). After the intervention, the distributions over the different types in OHS1 were not different compared to the period before the intervention.
In general, half of the incident reports were from GPs and the other half from medical nurses. Reporting chauffeurs were scarce. Concerning possible consequences for the patients, one-third of the incidents reported had the potential for permanent harm (κ = 0.63).
The intervention did not change type of reporters or the extent of potential harm of the incident reports.
Improvement measures
Qualitative analysis revealed that before the intervention the improvement measures were characterised by generically formulated recommendations about the incident, such as ‘GP should pay more attention to …’. Furthermore, after the CIRP had assessed the incident report, it was regularly advised that improvement measures should be developed locally. Often such measures had already been initiated on an ad hoc basis by the local management, when receiving this recommendation from the CIRP.
After the intervention, the improvement measures taken in OHS1 in 2007 and 2008 were formulated in more specific language and could be implemented much more easily than before the implementation of the LIRP. The managers thought that the fact that the measures originated from an incident from their own OHS location considerably facilitated implementation. Examples of these measures were a protocol for informing and preparing GPs who were new in OHS1, and improving the strategy of quickly administrating patient data on the telephone.
Interestingly, the intervention in OHS1 had a general spin-off. It enabled the CIRP to stimulate the local-improvement initiatives not only in OHS1 but also in the other OHS locations of the collaboration. Moreover, the CIRP started focusing on trends and recurring problems instead of on incident reports only. For instance, a prospective risk analysis23 of the process of assigning home visits by the CIRP was triggered by several incident reports from different OHSs. In addition, a root cause analysis was performed in response to six incidents with tissue glue for small cut wounds, reported in the last 2 years from four OHSs. Both inquiries were started in 2007, and the resulting recommendations were implemented in all OHS locations, including the intervention OHS, 1 year later.
When comparing the local-improvement measures and the improvement measures initiated by the CIRP after the intervention, the local measures were implemented in a much shorter time frame.
DISCUSSION
Summary of main findings
The results from the present study indicate that a LIRP in a GPs' OHS with a fast track from incident report to improvement measures, yields many more incident reports than the CIRP in which reports are assessed in a longer cycle and outside the actual ‘working environment’.
In addition, locally initiated improvements seemed to be more practical and implemented more quickly.
Limitations of the study
The study may have some limitations. The participating OHS locations were not randomly but purposefully selected. The intervention was implemented in OHS1 because its location management had a positive attitude to incident reporting, which at that time was still a sensitive issue. It should be noted, however, that management of OHS1 was not involved in patient care during the out-of-hours operation. In addition, the actual reporters did not have any influence on the selection of the OHS for intervention. Furthermore, the control OHSs were comparable with the intervention OHS in geographical and socioeconomic status of the population, and also in organisational and patient features. Finally, the three OHSs had a comparable willingness to report incidents in the period before the introduction of LIRP. Therefore, it is unlikely that the selection of the intervention site seriously influenced the study findings.
Furthermore, because the number of incident reports in the control OHSs was small, the comparison of distributions concerning the type of incident reports, type of reporters, and potential harm should be cautiously interpreted.
Comparison with existing literature
To the authors' knowledge, this study is the first to experimentally compare two different procedures for reporting and analysing incidents in general practice. Other studies concerning the number and/or nature of incident reports in general practice have been mostly descriptive.5,7,20,24–30
The large increase in the number of incident reports in the first year after the introduction of the LIRP was followed by a smaller number of incident reports in the second year of follow-up. As the support of the study team was withdrawn in the second year of follow-up, this tailing off suggests that the team itself contributed to the effect of the intervention. However, the input of the research team was limited, and the increase could also be considered as a starting effect, as observed in many other implementation studies.31 Even so, in the second year of implementation, the number of reports in OHS1 still remained seven times higher compared to baseline.
In contrast to earlier publications, the present study concerns the reporting process up to and including the design and implementation of improvements in a GP setting. Closing the reporting cycle up to feedback and visible improvement actions is recognised as an important feature of effective incident reporting.32 Several publications have indicated that regular, timely, and meaningful feedback is important in an incident-reporting procedure.33–35 The increase of incident reports in 2007 and 2008 in the present study may be explained by these attributes of the LIRP. The fact that practical improvement measures were implemented more quickly in the LIRP than in the CIRP may also have contributed to this increase. Furthermore, the authors believe that the LIRP enabled the professionals to control the assessment of their incident reports, as the reports remained within their own OHS. This, together with the emphasis on systematic analysis and organisational learning,36 may have convinced caregivers to increase their level of incident reporting compared to the former CIRP.
Implementation of a LIRP is associated with extra costs for administration and analysis. Obviously, the benefits of the resulting improvement measures should outweigh these costs. When interviewed, management and staff indicated that the LIRP was feasible and improved patient safety (data not shown).
Implications for clinical practice and future research
This study suggests that the willingness to report incidents in a GP out-of-hours setting increases with a LIRP compared to a CIRP. It may also result in faster implementation of improvements. A central overview, however, would collate information from various LIRPs to identify trends, which would extend the opportunities for analysis and learning. This implies that the best way of dealing with incidents in order to learn from them, is to develop a system in which both central and local incident-reporting procedures are combined.
Acknowledgments
We would like to express our gratitude to the primary care professionals and the management of Primair Huisartsenposten (Primair Collaboration of GP out-of-hours services) in The Netherlands for their trust and for their cooperation to this study.
Notes
Funding body
SBOH (Dutch financer for GP vocational training institutes and GP trainees).
Ethical approval
Not applicable.
Competing interests
The authors have stated that there are none.
- Received March 31, 2010.
- Revision received May 11, 2010.
- Accepted July 5, 2010.
- © British Journal of General Practice, January 2011
