Abstract
Background Medication error is an important contributor to patient morbidity and mortality and is associated with inadequate patient safety measures. However, prescribing-safety tools specifically designed for use in general practice are lacking.
Aim To identify and update a set of prescribing-safety indicators for assessing the safety of prescribing in general practice, and to estimate the risk of harm to patients associated with each indicator.
Design and setting RAND/UCLA consensus development of indicators in UK general practice.
Method Prescribing indicators were identified from a systematic review and previous consensus exercise. The RAND Appropriateness Method was used to further identify and develop the indicators with an electronic-Delphi method used to rate the risk associated with them. Twelve GPs from all the countries of the UK participated in the RAND exercise, with 11 GPs rating risk using the electronic-Delphi approach.
Results Fifty-six prescribing-safety indicators were considered appropriate for inclusion (overall panel median rating of 7–9, with agreement). These indicators cover hazardous prescribing across a range of therapeutic indications, hazardous drug–drug combinations and inadequate laboratory test monitoring. Twenty-three (41%) of these indicators were considered high risk or extreme risk by 80% or more of the participants.
Conclusion This study identified a set of 56 indicators that were considered, by a panel of GPs, to be appropriate for assessing the safety of GP prescribing. Twenty-three of these indicators were considered to be associated with high or extreme risk to patients and should be the focus of efforts to improve patient safety.
- ambulatory care
- consensus
- general practice
- medication errors
- patient safety
- primary health care
- quality indicators
INTRODUCTION
Prescribing medications is one of the most powerful tools available to GPs in the prevention and treatment of disease, and alleviation of symptoms. Nearly 961.5 million community prescriptions are dispensed annually in England.1 However, medication-related adverse events arising as a result of primary care prescribing are an important source of patient morbidity, much of which could be prevented by the highest quality prescribing and medicines management.2 Prescribing-safety indicators are statements describing prescribing events that put the patient at risk of harm. Using these statements to audit or analyse clinical records is a powerful way for GPs to conduct focused and high yield-for-work audit activity on their own prescribing.
Worldwide, there have been many attempts over the past decade to develop prescribing indicators (mostly using consensus methods) for use in non-specific settings.3–11 Most of the indicator sets developed specifically for primary care originated in the UK.12–17 Other methods of analysing prescribing also exist, for example, prescribing analysis and cost tabulation (PACT) data and the Medication Appropriateness Index (MAI).18,19 There are obvious limitations with PACT data because of a lack of linkable clinical information, and the MAI is a time-consuming process involving detailed medical records analysis. With the development of methods for interrogating electronic medical records,20 there is now the opportunity in the UK to develop and use sophisticated indicators that can give a rapid assessment of the quality and safety of prescribing in individual practices using ‘plug and play’ software similar to that used in Quality and Outcomes Framework (QOF) assessment. This study aimed to update and expand, to whole-practice applicability, a set of prescribing-safety indicators (previously developed by some of the authors in 200915) for assessing the safety of prescribing of individual GPs in UK general practice. It also aimed to identify which of these indicators were associated with the greatest risk of harm to patients.
METHOD
Identification of indicators
An extensive literature review of tools for patient safety in general practice was used to source indicators. Two independent reviewers followed Cochrane guidelines for systematic reviews21 (for a full description of the search strategy see Appendix 1). Search terms in three stems (setting, topic, and output) were performed on the following databases: Embase, CINAHL, Pubmed, MEDLINE (Ovid 1996 onward), Health Management Information Consortium, and Web of Science on 1 November 2011. The output was too heterogeneous to apply meta-analysis techniques, so the study considered individual indicators using the following process. First, potential indicators that described a pattern of prescribing that could be hazardous and may put patients at risk of harm were identified and then the exclusion criteria shown in Box 1 were applied. New indicators found in the systematic review process were added to an existing set of 34 indicators published by the authors.15 The literature was re-reviewed particularly to identify new indicators published since 2009.
Box 1. Inclusion and exclusion criteria
Inclusion criterion
-
The indicator describes a pattern of prescribing that is potentially hazardous and may put patients at risk of harm.
Exclusion criteria
-
The indicator describes a pattern of prescribing that is so unusual in UK general practice that the yield is likely to be too low to justify inclusion in the indicators set.
-
Extraction of data required for the indicator (from general practice electronic health records) is unlikely to be feasible.
How this fits in
GPs have a very important role in improving patient safety by carefully prescribing and monitoring patients’ medicines. Nevertheless, many patients are put at risk, and some are harmed, as a result of hazardous prescribing in general practice. Assessing the safety of prescribing by GPs is an important feature of patient safety measures. This study updates the prescribing-safety indicators developed by the authors in 2009 for the Royal College of General Practitioners, and identifies which of these indicators are most likely to lead to harm to patients.
The indicators developed in this project were derived mostly from existing sources because of the considerable amount of work that has been done on prescribing-safety indicators in other countries and other settings. In addition to the sources for the Royal College of General Practitioners (RCGP) indicators,3–5 a number of new key papers were found that were previously unknown to the authors (Table 1).6–11,17 Over 600 prescribing indicators were reviewed against the inclusion and exclusion criteria, and against attributes of good indicators, including importance, validity, and feasibility of data collection using electronic health records. After removing duplicates, 34 original RCGP indicators and 37 new indicators were considered suitable for inclusion in round 1 of the subsequent consensus technique.
Source descriptors of indicators
There are various reasons why few candidate indicators were chosen from some of the sources. For example, many indicators from the Screening Tool of Older Person’s Prescriptions5 relate more to the appropriateness of prescribing than to safety. The Beers criteria3 relate more to US prescribing during the 1990s than they do to the current UK situation. Reasons for not selecting more indicators from new sources are listed in Table 1. Grey literature from patient safety organisations, such as the National Patient Safety Agency and the Agency for Healthcare Research and Quality, and targets from sources, such as the QOF and the National Service Framework for Older People, were also examined for potential indicators.
Defining the indicators and evidence base
Electronic searches of the literature were conducted and respected reference sources, such as the British National Formulary,22 Stockley’s Drug Interactions,23 and Martindale24 were drawn on. The supporting evidence base was summarised by a clinical pharmacist as a synopsis for each of the potential indicators and was also used to inform the rewording of indicators, or in some cases, generate new indicators. In most cases, indicators were defined according to the wording used in previously published studies and reports but, in some instances, this was altered either to make the indicators more relevant to UK general practice or to give more specific detail regarding the drugs and conditions covered. For eight of the new indicators, one or more variations in the text were produced; this resulted in a total of 48 indicator statements for the 37 indicators. As an example (see next page), the original indicators (a) and (b) (drawn from Guthrie et al ’s17 and Zhan et al ’s10 indicators) were presented to the panel as the two original statements and also in a format suggested by the research team (c). The panel eventually refined the indicator to version (d), which achieved consensus.
-
Tricyclic prescribed to a patient with heart failure.
-
Tricyclic prescribed to a patient with an arrhythmia, heart block or postural hypotension.
-
Amitriptyline at dose >75 mg prescribed to a patient with the above cardiac problems.
-
Amitriptyline at dose >75 mg prescribed to a patient with heart failure, arrhythmia, heart block or postural hypotension.
Consensus process
The RAND Appropriateness Method, which is an established approach for the development of health indicators,25–28 was used to select the most appropriate indicators. This method seeks to combine scientific evidence with the collective judgement of experts: a consensus opinion is derived from a group, with individual opinions aggregated. An attempt was made to re-recruit the 12 panel members who met and agreed on the 2009 RCGP indicators15 to achieve consistency in reconsidering previously published indicators for a new purpose. Nine members of the original panel participated and were joined by three local GPs recruited from the Vale of Trent Faculty of the RCGP. The following factors were taken into account when selecting panellists: professional background (practising GPs), employment status (a range of different types of GP, such as, partners, salaried, and so on), sex, geographic location (GPs represented each of the four countries of the UK), and professional roles (some GPs with experience of working as appraisers and/ or assessors of GPs). It was estimated that each member of the panel committed at least 3 days of work to the consensus-building exercise; they were partially reimbursed for their time. A further exercise was also conducted to assess the risk to patients associated with the indicators that were selected by the RAND panel (two additional GPs with expertise in prescribing who were used to replace two panellists who unfortunately could not participate in this later exercise).
Data analysis
This study adhered to the RAND Appropriateness Method25 by conducting a two-round consensus process. In round 1, which was conducted by email in July 2012, panel members were asked to consider each indicator on its own merits using the summarised evidence for each as well as their own experience as practitioners. Panellists were asked to consider separately the suitability of each indicator for assessing the safety of individual prescribers and for assessing the safety of the practice as a whole. In round 2, panellists met for a 1-day face-to-face meeting in July 2012, where under the chairmanship of two moderators (one with extensive experience in RAND panel methodology), they discussed each indicator in turn as a group and then re-rated the indicators on individual rating sheets. These round-2 rating sheets included the panellist’s own rating on round 1, and for comparison, presented the frequency distribution of ratings of all panellists (anonymised) and the overall panel median rating. During round 2, panellists also had the option to propose alternative wording for indicators, which they would then refine by consensus decision. In both rounds, panellists were asked to rate each indicator on a nine-point scale. A rating of 1 meant that it would be extremely inappropriate to use the indicator, whereas a rating of 9 meant its use would be extremely appropriate. The overall panel median ratings were as follows: 1–3: inappropriate; 4–6: equivocal, or unsure of appropriateness; and 7–9: appropriate. The level of consensus within the panel for each scale for each indicator was also calculated. Agreement signified that no more than 20% of panellists’ ratings were outside the same 3-point region (that is, 1–3, 4–6, 7–9) as the observed median (that is, for a 12-person panel no more than two ratings outside this 3-point region). This method was identical to the one used in the authors’ previously published research.15 Results are presented for the final (round 2) ratings only.
A subsequent Delphi methodology exercise asked the panellists to rate the potential harm to patients and the likelihood of hazardous prescribing for those indicators that had been considered appropriate by the RAND panel. A single round was conducted by email. Harm was rated on a 5-point scale that ranged from 1 (insignificant) to 5 (catastrophic); likelihood was rated on a 5-point scale that ranged from 1 (rare) to 5 (almost certain). Panellists were asked to use the information provided in a background document and their own experience as clinicians to rate the indicators. They were also provided with information on how commonly the drugs were prescribed in England in 2011 (prescriptions dispensed per 1000 patients). Panellists were invited to provide comments, but were asked to rate the indicators as they were written. Harm and likelihood scores from each panellist were multiplied to put each indicator into one of four risk categories.29 The risk categories were: 1–3 (low risk), 4–6 (moderate risk), 8–12 (high risk), and 15–25 (extreme risk). Indicators were considered high or extreme risk when the median risk category for that item was high or extreme and 80% or more of the participant scores were in the same risk category as the median.
RESULTS
Figure 1 shows the steps taken in arriving at the final set of indicators and the criteria that were applied to select the indicators at each step. One hundred indicators were considered in round 2 of the exercise but two of these were removed because of unnecessary duplication; the results from the rating exercise are available from the authors on request. A total of 42 indicators were not included in the set; of these, 11 indicators were given a panel median rating of 1–3, 25 were given a rating of 4–6, and 6 achieved a median of 7–9, but there was a lack of consensus among the raters (as defined above). Therefore, 56 prescribing safety indicators were included in the final set as they were considered appropriate for assessing the safety of prescribing of individual GPs and/or general practices (overall panel median rating of 7–9, with agreement). These indicators are shown in Table 2 (further details are available from the authors). All but three of these indicators were considered appropriate for assessing the safety of individual GPs and general practices. Of the 56 indicators considered appropriate, five were suggested by the panel in round 2. Thirty-one of the 34 existing RCGP indicators were ratified for inclusion; the remaining 25 indicators were newly sourced.
Number of indicators at each step of the consensus process.
Indicators rated as appropriate for assessing the safety of prescribing in general practice
Of the 56 indicators in Table 2, 19 were in the high risk category and four were in the extreme risk category, with 80% or more of the participants rating these items as either high risk or extreme risk. High risk and extreme risk indicators are fairly evenly distributed across the major categories. The median harm, likelihood, and risk scores for all 56 indicators and the percentage of responders who provided a score in the same risk category as the median are available from the authors on request.
DISCUSSION
Summary
Fifty-six prescribing-safety indicators were identified as appropriate for use in general practice. Violation of any of these indicators suggests a potential patient safety problem. These indicators covered hazardous prescribing across a range of therapeutic indications, hazardous drug–drug combinations, and inadequate laboratory test monitoring. Twenty-three of these indicators posed a high or extreme risk of harm to the patient.
Strengths and limitations
The strengths of this study include the wide range of sources and use of a formal literature review to identify prescribing-safety indicators. A large number of indicators (over 600) were reviewed and each indicator was presented to the panel accompanied by an evidence-based summary written by a clinical pharmacist. To construct the evidence-based summaries electronic searches of the literature were conducted and respected reference sources were drawn on.22–24 This study also adhered to a validated systematic consensus method for developing appropriateness scenarios. Consistency in ratings was achieved by successfully recruiting most (9 out of 12) of the panel members who had originally rated the 2009 RCGP indicators.15 There was also overlap (9 out of 11) between the panel members who provided the risk ratings and those who participated in the RAND exercise. In general, the method appears to be highly reliable as 31 of the 34 indicators that were previously presented in the 2009 RAND Appropriateness Method exercise were still given ratings high enough for inclusion. Three indicators from the earlier set were not included because they received ratings of 6–7. The degree of risk of harm to patients associated with each indicator was assessed and this is potentially helpful in terms of identifying which indicators may be most important to focus on when trying to identify patients at greatest risk within a general practice. However, a limitation is that the views of the GP panel members were sought just once in relation to this aspect of the study.
Comparison with existing literature
New indicators found in the systematic review process were added to an existing set of 34 indicators previously published by the authors,15 to update the set. Owing to constant updates in the field of prescribing indicators, a large set of indicators have been published by a team in Scotland since the authors’ systematic review was undertaken and these will be considered in future work.30 In contrast to a 2013 study29 of secondary care prescribing indicators (in which 80 out of 109 indicators (73%) were considered high or extreme risk) a smaller proportion were considered high or extreme risk (23 out of 56 or 41%) in this study. This is likely to be due to differences in the indicators and differences in risk of harm between primary and secondary care.
Implications for research and practice
The results from this study have implications for future research and practice. The authors plan to test the acceptability, feasibility, reliability, and validity of the prescribing indicators in a sample of English general practices. This will aim to determine whether there are any problems with implementing the indicators in general practices and will enable the description of variations in the safety of prescribing between practices. An intervention study will also be conducted to determine whether these indicators, as part of a larger patient safety toolkit, can be used to improve patient safety. The indicators might equally well be used for practice learning, revalidation or audit purposes (see article series in Prescriber 31–36 for examples of how this can be accomplished).
Work will also be conducted to determine whether each indicator can be translated into a computer query capable of assessing the prescribing safety of individual GPs. The use of these queries is being tested on GP computer systems to examine whether the use of computerised prescribing-safety indicators improves prescribing performance. The challenge for future work would be to prove that this does improve patient outcomes. This goal has already been achieved in the PINCER trial16 for several of the indicators.
Acknowledgments
We wish to thank the following GPs who participated in rating the indicators: Olesya Atkinson, William Beebe, Beth Coward, Maureen Crawford, Martin Duerdens, Simon Hurding, Adam Liew, Rosalind Mills, Julie Osborne, Lindsay Pope, Beth Rimmer, Mohammed Sharif, Bijoy Sinha, and Arnold Zermansky.
Appendix
Literature review search strategy
Notes
Funding
National Institute for Health Research School for Primary Care Research, Project Number 113.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors have declared no competing interests.
Discuss this article
Contribute and read comments about this article: www.bjgp.org/letters
- Received September 27, 2013.
- Revision received October 28, 2013.
- Accepted January 2, 2014.
- © British Journal of General Practice 2014