Abstract
Background In the UK, a process of revalidation is being introduced to allow doctors to demonstrate that they meet current professional standards, are up-to-date, and fit to practise. Given the serious risks to patients from hazardous use of medicines it will be appropriate, as part of the revalidation process, to assess the safety of prescribing by GPs.
Aim To identify a set of potential prescribing-safety indicators forthe purposes of revalidation of individual GPs in the UK.
Design and setting The RAND Appropriateness Method was used to identify, develop, and obtain agreement on the indicators in UK general practice.
Method Twelve GPs from across the UK with a wide variety of characteristics assessed indicators for appropriateness of use in revalidation.
Results Forty-seven safety indicators were considered appropriate for assessing the prescribing safety of individual GPs forthe purposes of revalidation (appropriateness was defined as an overall panel median score of ≥7 (on a 1-9 scale), with no more than three panel members rating the indicator outside the 3-point distribution around the median]. After removing indicators that were variations on the same theme, a final set of 34 indicators was obtained; these cover hazardous prescribing across a range of therapeutic areas, hazardous drug-drug combinations, prescribing with a history of allergy, and inadequate laboratory-test monitoring.
Conclusion This study identified a set of 34 indicators that were considered, by a panel of 12 GPs, to be appropriate for use in assessing the safety of GP prescribing forthe purposes of revalidation. Violation of any of the 34 indicators indicates a potential patient-safety problem.
INTRODUCTION
Medication prescription is one of the most powerful tools for GPs in the prevention and treatment of disease and the alleviation of symptoms. Nearly800 million prescriptions are dispensed annually in the community in England.1 However, medication-related adverse events are an important source of patient morbidity, many cases of which could be prevented by the highest-quality prescribing and medicines management.2-5
Revalidation is a single, integrated process that allows doctors working in the UK to demonstrate that they meet current professional standards, are up-to-date, and fit to practise, by providing supporting information in a revalidation portfolio.6 Pilots of revalidation are currently ongoing. For GPs, the Royal College of General Practitioners (RCGP) is proposing standards and methods that must be approved by the General Medical Council — these include prescribing-safety indicators. Given the importance of prescribing by GPs, it is appropriate that potential indicators of the quality and safety of prescribing are considered in any scheme for revalidation.
There have been many attempts over recent years to develop prescribing indicators in the UK,7 such as those based on the interrogation of prescriptions issued by GPs (for example, using prescribing analysis and cost tabulation data8). Although potentially useful for analysing prescribing patterns, these data are rarely linked to diagnoses and patient characteristics, and so have limitations when assessing quality and safety. Other indicators have required very detailed analysis and assessment of clinical records (for example, the medication appropriateness index9), which would not be feasible for the large-scale assessment of all GPs in the UK.
The evidence base for developing quality indicators for safe prescribing is limited; however, systematically combining available evidence with expert professional opinion using a consensus methodology can create quality indicators in areas where it would not otherwise be possible.10 Moreover, with developments in methods for interrogating electronic medical records, there is now the opportunity in the UK to develop and use sophisticated indicators that can give an assessment of the quality and safety of prescribing by individual GPs.
This study aimed to identify a set of potential prescribing-safety indicators for the purposes of revalidation of individual GPs in the UK, contextualised by the wider implementation of revalidation.
METHOD
Identification of indicators
Potential indicators that described a pattern of prescribing that could be hazardous and may put patients at risk of harm were identified. Potential indicators that were not suitable for assessing the prescribing of individual doctors and where the prospects were low for extracting relevant data from GP clinical computer systems were excluded. The inclusion and exclusion criteria are shown in Box 1.
How this fits in
GPs have a critically important role in patient safety through the judicious prescription, and careful monitoring, of patients' medicines. Nevertheless, many patients are put at risk, and some are harmed, as a result of hazardous prescribing in general practice. Therefore, assessing the safety of prescribing by GPs is an important facet of any system for assessing fitness to practise. One way of doing this is by using prescribing-safety indicators, which have been developed in other areas of medicine. This article describes the development of a set of prescribing-safety indicators for use in general practice, with a specific focus on their use in revalidation in the UK.
Box 1. Inclusion and exclusion criteria
Inclusion criterion
The indicator describes a pattern of prescribing that is potentially hazardous and may put patients at risk of harm.
Exclusion criteria
The indicator describes a pattern of prescribing that could not easily be attributed to a single doctor
The indicator describes optimal prescribing for long-term conditions where prescribing decisions are likely to be made overtime (potentially by more than one prescriber), rather than at a single consultation.
The indicator describes a pattern of prescribing that is so unusual in UK general practice that the yield is likely to be too low to justify inclusion in the indicators set.
Extraction of data required for the indicator (from general practice electronic health records) is unlikely to be feasible.
Multiple sources of information were used to identify potential indicators;11-24 these are outlined in Table 1. These sources were identified by one of the researchers with the help of a reference group between August 2008 and March 2009. The indicators developed in this project were derived mostly from existing sources because of the considerable amount of work that has been done on prescribing-safety indicators in other countries and other settings. 11-24 Over 400 prescribing indicators were reviewed against the inclusion and exclusion criteria and against attributes of good indicators, including importance, validity, and feasibility of data collection using electronic health records.10,25
Table 1 Sources of the reviewed prescribing indicators
After removing duplicates, 50 indicators were considered suitable for inclusion in round one of the subsequent consensus technique. Of these, 44 came from a least one of the sources highlighted in Table 1; four came only from the British National Formulary (BNF)14 and two only from the draft design specification for NHS IT systems.15
Defining the indicators and evidence base
In most cases, indicators were defined according to the wording used in previously published studies and reports but, in some instances, this was altered either to make the indicators more relevant to UK general practice or to give more specific detail regarding the drugs and conditions covered. For 13 of the indicators, one or more variations in the text were produced; this resulted in a total of 68 indicatorstatements for the 50 indicators. As an example, although the prescription of β-blockers is contraindicated in asthma, there is evidence to suggest that in various types of heart disease, such as heart failure, the benefits may outweigh the risks; therefore, a variation on an asthma/β-blocker indicator would be to exclude patients with cardiac conditions where benefits of β-blockers may outweigh the risks.
Electronic searches of the literature were conducted and respected reference sources, such as the BNF,14 Martindale: The Complete Drug Reference,26 and Stockley's Drug Interactions,27 were drawn on. Indicators were also shown to the team responsible for developing the content of the BNF, who provided additional background evidence. The supporting evidence base was summarised in an evidence-based synopsis for each of the potential indicators.
The reasons why the different types of indicators did not yield many candidate indicators from the sources included are shown in Table 1. For example, many Assessing Care of Vulnerable Elders11 indicators developed in the US, relate to process measures rather than prescribing decisions that can be attributed to one GP; likewise, some of the Beers criteria13 and indicators of the Screening Tool of Older Persons' potentially inappropriate Prescriptions (STOPP)24 relate more to the appropriateness of prescribing than safety.
The indicators were presented in the following categories:
A: cardiovascular and respiratory disease;
B: central nervous system (including analgesics);
C: anti-infective agents;
D: women's health and urinary disorders;
E: musculoskeletal;
F: hazardous co-prescriptions, interactions, and allergy; and
G: laboratory-test monitoring.
Consensus process
The RAND Appropriateness Method was used to select the most appropriate indicators. This method seeks to combine scientific evidence with the collective judgement of experts: a consensus opinion is derived from a group, with individual opinions aggregated into refined aggregated opinion.25,28,29 A 12-member panel was appointed. Two GPs with known expertise in therapeutics and one with experience of working with the British Medical Association's General Practitioners' Committee were purposely selected. The remaining nine panelists were identified by writing to all RCGP faculties in the UK in March 2009 to ask for volunteers, which resulted in 58 replies. The following factors were taken into account when selecting the remaining panelists:
professional background (practising GPs);
employment status (a range different types of GP, including partners, salaried GPs, a retainer-scheme GP, and a locum);
sex (an equal mix of males and females);
geography (GPs from each of the three countries of the UK); and
professional roles (some GPs with experience of working as appraisers/assessors of GPs).
It was estimated that each memberof the panel committed at least 3 days of work to the consensus-building exercise; they were partially reimbursed for their time.
Data analyses
This study adhered to the RAND Appropriateness Method by conducting a two-round consensus process. In round one, which was conducted by email in June 2009, panel members were asked to consider each indicator on its own merits using the summarised evidence for each as well as their own experience as practitioners.
In round two, panelists met for a 1-day face-to-face meeting in July 2009, where, under the chairmanship of two moderators, they discussed each indicator in turn as a group and then re-rated the indicators on individual rating sheets. These round-two rating sheets included feedback from the first round, presenting the frequency distribution of ratings of all panelists across the 9-point scale, the overall panel median rating, and, for each panelist, a reminder of how they had rated each indicator in round one. Although panel members were made aware of their own ratings from the first round, the scores from other panel members were anonymised.
During round two, panelists were also asked to consider whether they wished to propose an alternative wording for each indicator or propose any additional indicators. This resulted in wording changes for 21 indicators and two new ones. Each of the indicators — whether original, revised, or new — was scored by each of the panel members.
In both rounds, panelists were asked to rate each indicator on a 9-point integer scale. A rating of 1 meant that, in their opinion, it would be extremely inappropriate to use the indicator, whereas a rating of 9 meant its use would be extremely appropriate. The overall panel median ratings were as follows:
Using the consensus technique, there was another aspect to the rating process, namely, the level of agreement within a panel for each scale for each indicator. Agreement signified that 80% of panelists' ratings were within the same 3-point region (that is, 1-3, 4-6, 7-9), as the observed median. To be included in the final recommended set of indicators, therefore, ‘appropriate’ was defined as an overall panel median score of >7, with no more than three panel members rating the indicator outside the 3-point distribution around the median; for example, if the median score was 7, no more than three panel members gave a score <6 or >8. Results are presented for the final (round-two) ratings only.
RESULTS
Ninety-two indicators were considered in round two of the exercise; the full results are shown in Appendix 1. Forty-seven prescribing-safety indicators were considered appropriate for assessing the prescribing safety of individual GPs for the purposes of revalidation (overall panel median rating of 7-9 with agreement); these are shown in bold in Appendix 1). These consisted of 32 originally worded indicators, along with 13 variations on these and two that were suggested by the expert panel during round two. Six indicators were rated inappropriate for assessing the prescribing safety of individual GPs for the purposes of revalidation (overall panel rating of 1-3 with agreement). The remaining indicators were rated as equivocal.
After removing indicators that were variations on the same theme, 34 that were rated as appropriate for use in assessing the prescribing safety of individual GPs for the purposes of revalidation were identified; these are shown in Box 2. Variations on indicators that were either most strongly rated by panelists or were most comprehensive were selected — for example, panelists rated indicators involving benzodiazepines more strongly where these indicators also contained Z-drugs.
Box 2. Indicators rated as valid for assessing the prescribing safety of individual GPs
A: Cardiovascular and respiratory disease
1. Beta-blockerto a patient with asthma (excluding patients who also have a cardiac condition, where the benefits of beta-blockers may outweigh the risks
2. Short-acting nifedipine (excluding patients with Raynaud's disease)
3. Digoxin at a dose >125 μg daily in a patient with renal impairment (for example, CKD3 or worse)
4. Digoxin at a dose of >125 μg daily for a patient with heart failure who is in sinus rhythm
5. Diltiazem orverapamil in a patient with heart failure
6. Aspirin at a dose >75 mg daily for ≥1 month in a patient aged >65 years
7. Long-acting beta-2 agonist inhaler to a patient with asthma who is not also prescribed an inhaled corticosteroid
B: Central nervous system (including analgesics)
8. Aspirin to a child aged ≤16 years
9. Metoclopramide or prochlorperazine in a patient with Parkinson's disease
10. Benzodiazepine or Z-drug for ≥21 days in a patient aged >65 years who is not receiving benzodiazepines orZ-drugs on a long-term basis
11. Initiation of benzodiazepine or Z-drugs basis for ≥21 days in a patient >65 years with depression
C: Anti-infective agents
12. Mefloquine to a patient with a history of convulsions
D: Women's health and urinary disorders
13. Combined hormonal contraceptive to a woman with a history of venous or arterial thromboembolism
14. Oral ortransdermal oestrogens to a woman with a history of breast cancer
15. Oral ortransdermal oestrogen without progesterone in a woman with an intact uterus
16. Combined hormonal contraceptive to a woman aged ≥35 years who is a current smoker
17. Combined hormonal contraceptive to a woman with body mass index ≥40
E: Musculoskeletal
18. NSAID, without co-prescription of an ulcer healing drug, to a patient with a history of peptic ulceration
19. NSAID in a patient with heart failure
20. NSAID in a patient with chronic renal failure (for example, CKD3 orworse)
21. Long-term (>28 days) NSAID (except for ibuprofen ≤1200 mg daily) in a patient aged >65 years
F: Hazardous co-prescriptions, interactions, and allergy
22. Warfarin in combination with an oral NSAID
23. Phosphodiesterase type-5 inhibitor (for example, sildenafil) to a patient who is also receiving a nitrate or nicorandil
24. Clarithromycin or erythromycin to a patient who is also receiving simvastatin, with no evidence that the patient has been advised to stop the simvastatin while taking the antibiotic
25. Potassium salt or potassium sparing diuretic (excluding aldosterone antagonists) to a patient who is also receiving an ACE inhibitor or angiotensin II receptor antagonist
26. Verapamil to a patient who is also receiving a beta-blocker drug
27. Penicillin-containing preparation to a patient with a history of allergy to penicillin
G: Laboratory test monitoring
28. Warfarin to a patient without a record of international normalised ratio having been measured within the previous 12 weeks (excluding patients who self monitor)
29. Amiodarone without a record of liver function being measured in the previous 9 months
30. Amiodarone without a record of thyroid function being measured within the previous 9 months
31. ACE inhibitor or angiotensin II receptor antagonist without a record of renalfunction and electrolytes being measured prior to starting therapy
32. Lithium without a record of a lithium level being measured within the previous 6 months
33. Methotrexate without a record of a full blood count within the previous 3 months
34. Methotrexate without a record of liver function having been measured within the previous 3 months
DISCUSSION
Summary
Thirty-four prescribing-safety indicators were identified as appropriate for use in assessing the prescribing safety of individual GPs for the purposes of revalidation. These cover hazardous prescribing across a range of therapeutic areas, hazardous drug-drug combinations, prescribing with a history of allergy, and inadequate laboratory-test monitoring.
Strengths and limitations
The strengths of the study include the wide range of sources used to identify prescribing-safety indicators as well as the involvement of both a reference group and staff editors from the BNF to validate aspects of the work and the evidence base. Each indicator was accompanied by an evidence-based summary. The panel consisted of GPs with a wide range of characteristics. Although over 400 potential prescribing indicators were reviewed, only those appropriate for use in the assessment of the prescribing safety of individual GPs for the purposes of revalidation were included. This study also adhered to a validated systematic consensus method for developing appropriateness scenarios.29
The indicators were not defined so they could be immediately converted into reliable and unambiguous queries for running on GP computer systems, as with the PINCER trial.21 In addition, the indicators were not phrased in such a way as to take account of a doctor's volume of prescribing (whereby instances of hazardous prescribing might be considered as a proportion of overall numbers of prescriptions).
For each of the potential indicators, the supporting evidence base was summarised by undertaking electronic searches of the literature and drew on respected reference sources, such as the BNF,14 Martindale: The Complete Drug Reference,26 and Stockley's Drug Interactions.27 Although it is possible that more extensive literature reviews would have identified further evidence or other sources of potential prescribing-safety indicators, the involvement of a reference group and consultation with the editors of the BNFmitigate against this.
Comparison with existing literature
It was felt that the modified Beers criteria13 and STOPP indicators24 would be of particular relevance to UK general practice, although fewer than half of the indicators from these sources were considered appropriate for use in revalidation. There are a number of reasons for this, such as the inclusion and exclusion criteria utilised, the perceived clinical importance of the indicators, and the focus on indicators for the purposes of revalidation. Nevertheless, the study shows the importance of giving careful consideration to the purpose forand setting in which they are planned to be used.
Implications for research and practice
This study has highlighted a number of issues that have implications for future research as well as the development and application of prescribing-safety indicators. Ongoing work will now determine whether each indicator can be translated into a computer query capable of reliably assessing the prescribing safety of individual GPs; the use of these queries will then be tested on GP computer systems. If implemented, it will be important to determine whether the use of prescribing-safety indicators improves prescribing performance and patient outcomes.
There are other implications that must be noted. The evidence base for most of the indicators is weak. This is, perhaps, unsurprising given that clinical trials focus on demonstrating the efficacy of interventions, rather than their safety; nevertheless, few of the indicators that were identified would fulfil the evidence required for inclusion in, for example, the Quality and Outcomes Framework.23,30 Building the evidence base would mean conducting further research on the harms associated with potentially hazardous prescribing and inadequate medication monitoring, as well as the extent to which avoiding/correcting hazardous prescribing and inadequate monitoring improves patient safety.
In addition, although several of the sources used took account of the most common and important medication errors and/or preventable adverse drug events in general practice,15,21,22 future pharmacoepidemiological studies might help to provide even better information for the development of prescribing-safety indicators.
It will be important to consider the ways in which the prescribing-safety indicators might be used. The primary purpose of this study was their use in revalidation, but the indicators could also be used for the purposes of audit, GP appraisal, and in intervention studies similar to the PINCER trial.21 Encouraging the use of the indicators for improving the safety of prescribing will be important.
This study identified a set of 34 indicators that were considered, by a panel of 12 GPs, to be appropriate for use in assessing the prescribing safety of GPs forthe purposes of revalidation. Violation of any of these indicators points towards a potential patient-safety problem.
Acknowledgments
We thank the GPs who took part in the consensus process; the administrative staff at the Royal College of General Practitioners; Clare Randall for administrative support at the University of Nottingham; Malcolm Campbell, Mike Pringle, Ian Purves, Martin Roland, and Martin Shelley, who were members of the reference group; and Colin Hunter and Nigel Sparrow who gave helpful advice throughout the course of the project.
