Abstract
Background Exception reporting allows practices to exclude eligible patients from indicators or an entire clinical domain of the Quality and Outcomes Framework (QOF). It is a source of contention, viewed by some as a ‘gaming’ mechanism.
Aim To explore GP and practice staff views and experiences of exception reporting in the QOF.
Design of study Qualitative semi-structured interviews.
Setting Interviews with 24 GPs, 20 practice managers, 13 practice nurses, and nine other staff were conducted in 27 general practices in the UK.
Method Semi-structured interviews, analysed using open explorative thematic coding.
Results Exception reporting was seen as a clinically necessary part of the QOF. Exempting patients, particularly for discretionary reasons, was seen as an ‘exception to the rule’ that was justified either in terms of practising patient-centred care within a framework of population-based health measures or because of the poor face validity of the indicators. Rates in all practices were described as minimal and the threat of external scrutiny from primary care trusts kept rates low. However, GPs were happy to defend using discretionary exception codes for individual patients. Exception reporting was used, particularly at the end of the payment year, to meet unmet targets and to prevent the practice being penalised financially. Overt gaming was seen as something done by ‘other’ practices. Only two GPs admitted to occasional inappropriate exception reporting.
Conclusion Exception reporting is seen by most GPs and practice staff as an important and defensible safeguard against inappropriate treatment or over-treatment of patients. However, a minority of practitioners also saw it as a gaming mechanism.
INTRODUCTION
Since 2004, approximately 25% of GPs' income in the UK has been based on meeting targets relating to quality indicators as part of the Quality and Outcomes Framework (QOF) pay-for-performance scheme.1 The indicators within the QOF are evidence based but involve making generalisations about people based on condition-specific symptoms, rather than treating patients as individuals.2 Adherence to the voluntary indicator scheme is not a substitute for clinical judgement and focusing on a patient's overall needs in the consultation.3
The indicators within the QOF are population-based measures underpinned by principles of quality measurement appropriate to ‘average’ patients with single conditions, such as diabetes, who are consulting under ‘average’ circumstances.4 Intentionally, they take no account of comorbidities or an individual's other social and medical contexts.5 To accommodate individual patient circumstances and choices, and the fact that not all patients fit easily into population averages, exception reporting is an integral part of the QOF, allowing practices to exclude eligible patients from the denominator of an indicator or an entire clinical domain.
Exception reporting was part of the QOF from 2004, but data on its use have only been collected since 2005. Rates have been steady at around 5%, but there is wide variation between indicators and practices.6 There are two types of exception reporting: non-discretionary and discretionary (Box 1).
Box 1 The two types of exception reporting and their criteria
Non-discretionary
Patients who refuse to attend an appointment and who have been invited on at least three occasions during the preceding 12 months
Patients who are newly diagnosed, or those recently registered with the practice, who should have measurements made within 3 months and delivery of clinical standards within 9 months (for example, blood pressure or cholesterol measurements within target levels)
Patients who do not agree to investigation or treatment (informed dissent) and have this recorded in their medical records
An investigative service or secondary care service is unavailable
Discretionary
Patients for whom it is not appropriate to review the chronic-disease parameters due to particular circumstances (for example, extreme frailty, terminal illness)
Patients who are on maximum tolerated doses of medication whose levels remain suboptimal
Patients for whom prescribing a medication is not clinically appropriate (for example, those with an allergy or who have experienced an adverse reaction to a prescribed medicine)
Patients who have not tolerated a medication
Patients who have a supervening condition that makes the treatment of their condition inappropriate (for example, cholesterol reduction in a patient who has liver disease)
There has been previous research on practice staff's views of the QOF,7–10 as well as quantitative analysis of the extent of exception reporting across England and its relationship to achievement.11 There are also data from Scottish practices that the QOF may provide ‘perverse incentives’, as practices that had lower levels of achievement in 2004–2005 were slightly more likely to except patients – and, therefore achieve more points – in 2005–2006.12 In practices that were below indicator thresholds in 2004–2005, 0.87% of patients might have been inappropriately exception reported in 2005–2006.12
There has been speculation about the place and value of exception reporting from both practitioners and the Department of Health.13–15 However, to the authors' knowledge, there has been little empirical research exploring the experiences and opinions of primary care staff and GPs' views of exception reporting in the QOF.15 This is an important omission as exception reporting is a source of contention, viewed by some as a ‘gaming’ mechanism.13 This article describes an in-depth exploration of GPs' and practice staff's views, understanding, concerns, and experiences of exception reporting.
METHOD
Sample and study participants
Practices were recruited as part of a wider ongoing process that involves piloting potential quality indicators for the QOF.1 Piloting involves the recruitment of separate cohorts of representative practices in 12 geographically disparate primary care trust (PCT) areas of England, and two practices each in Northern Ireland, Scotland, and Wales. Participants for this study were recruited from 28 practices from the first cohort of pilot practices (October 2009–March 2010). This English pilot cohort included practices that were nationally representative in terms of size, QOF achievement, Index of Multiple Deprivation (Table 1), and the use of different IT systems.
National representativeness of first English pilot cohort.a
Participants were nominated from within those practices taking part but, to be eligible for interview in the study, had to have played a leading role in the practice's day-to-day engagement with the QOF. Sixty-six general practice professionals and staff across 27 of the pilot practices were interviewed; interviews lasted around 45 minutes. The participants consisted of 24 GPs (from 23 practices), 20 practice managers (from 20 practices), and 13 nurses (from 11 practices). Nine other staff (from eight practices) who were, for the most part, IT staff or medical notes summarisers were also interviewed. Seven of the GPs were female and 17 were male; all were principal partners. Two-thirds of the GPs worked full time. All nurses were female, with two-thirds working part time.
How this fits in
Exception reporting is a source of contention within the Quality and Outcomes Framework. The indicators are evidence based but do not take into account patients' comorbidities or social and medical contexts. Exception reporting is an important and clinically necessary part of the QOF. It is used as a clinical safeguard when accounting for the complexities of primary health care and GPs' wishes to practice holism and act as patient advocates.
Data collection
Semi-structured interviews were conducted in the participants' own general practice workplaces between April and May 2010. An interview topic guide, based on the objectives of the study, was used (Box 2).
Box 2 Interview topic guide
What is your general view of exception reporting within the Quality and Outcomes Framework?
Should there be discretionary and non-discretionary exception reporting? (NB: Box 1 taken as a prompt)
Who do you exception report and why? Do you have practice or clinical rules or do you decide individually by clinician? Is it indicator or condition specific?
Data analysis
All interviews were digitally recorded and fully transcribed. The study was explorative, not theoretical, so open, rather than axial or selective, coding was used.16 Each transcript was read and coded separately by two of the authors and a preliminary inductive coding frame constructed.
Transcripts were coded inductively without knowing what would be found when the data analysis began. Upon completion of coding, those codes that had common elements were merged to form categories; this was to ensure that the analysis of the transcripts reflected the recurring and representative themes found.17 Disconfirming evidence was actively sought throughout.18
The findings are based on a synthesis of all the interviews in this study. All statements are based on the reports of multiple interviewees. The quotations were chosen on the grounds of representativeness.
RESULTS
There were three key themes in the data related to exception reporting:
reasons for exception reporting;
the level and appropriateness of exception reporting; and
the threat of external scrutiny on behaviour.
Reasons for exception reporting
All participants were asked whether their practice had any formal policy or underlying ground rules for exception reporting eligible patients from the QOF. No practice had a formal, written practice policy on exception reporting, but most focused on two key principles. The first of these related to adherence to the external national guidance rules on (non-discretionary) exception reporting19 as well as a parallel, internal, practice-specific focus reflecting on the suitability of individual patients for discretionary exclusion.
The ‘externally set’ rules particularly related to non-discretionary criteria, such as patients not responding to or refusing three invitations to attend a review and those not agreeing to an investigation or treatment (informed dissent):
‘We don't exception report unless we can avoid it. It's very much [a] last resort. If a patient clearly doesn't respond to our efforts to get them in, then they will be exception coded.’ (practice manager [PM] 6)
‘… if a patient has been written to on at least three occasions, and the patient does not attend at that point, then we can consider exception reporting. Secondly, if a patient clearly indicates in person they do not wish to be involved, we will exempt the patient.’ (PM 8)
These exception codes were often entered onto to the practice system by employed non-clinical practice staff.
Given that not all indicators were equally relevant to the needs of each individual patient on a given register, the second underlying principle related to an assertion that exception reporting was an integral part of the clinical judgment required in implementing indicators. This use of clinical judgment was expressed in two ways, one of which was through the use of specific ‘discretionary’ exception rules, such as patients being on maximal tolerated therapy. Participants also spoke of how exception reporting allowed them to focus on an individual patient's circumstances, contexts, or choices; for example, where a patient was old and frail or had multiple morbidities. This use of discretionary clinical judgment exception codes was always decided on, and almost always entered, by GPs.
A significant minority of GPs also stated that they were more likely to exception report indicators with poorly perceived face validity. This was because the care/treatment element of the indicator was felt to be contrary to their role as the patient's advocate and, in their clinical judgement, not relevant to individual patient-centred care:
‘I think you do need scope to exception report patients because people are really quite individual and unique, and I think, you know, in the end there is always an element of clinical judgment.’ (GP 29)
‘There is a place for exception reporting within [the] QOF. Discretionary versus non-discretionary, I think you probably have to keep both because of the complexities of the whole agenda. There are some areas where it needs to be discretionary … There are patients out there that just do not fit into this very nice, structured approach.’ (GP 24)
Clinical examples were primarily focused on indicators for depression and chronic renal failure:
‘The depression indicators with the PHQ [Patient Health Questionnaire], I think there's quite a strong feeling among the profession that that's been quite counter-productive to patient care … I think it's important that it enhances care for patients rather than detracts from patient care and these things we're doing actually reflect that. So it's not just a paper exercise, there is actually something that's going to add to quality really’. (GP 6)
A significant minority of participants referred to societal issues – such as the effect of deprivation on patient compliance, health-seeking behaviour, and morbidity – that related to particular groups of patients in their practice:
‘If you've got practices in very deprived areas they're going to have a lot more patients who won't attend …’ (GP 13)
Patients with, for example, serious mental illness were described as often consulting opportunistically, rather than at specific appointment times:
‘But mental health are particularly difficult because they're not the sort of patients you can say “come back next week”.’ (GP 19)
One family doctor spoke specifically about how unfair it would be to remove exception reporting from practices in areas of deprivation, and another made the same point with regard to patients who often don't attend when specifically invited to.
The level and appropriateness of exception reporting
Most participants emphasised that their own practice had low rates of exception reporting and said it was ‘not used lightly’ (PM 24) or that it was used ‘minimally’ (PM 26). Other responders stated that it was used ‘conservatively’ (GP 18), ‘ only when necessary’ (PM 17), or as ‘very much [a] last resort’ (PM 6) and in ‘extreme cases’ (PM 3). They felt that each and any use had to be defensible:
‘The only rule I think we have is that we feel that every exempt reporting needs to be defendable. So if you ask me why did you exempt this patient I should have an answer … even if you ask me 3 years later I should be able to look at a note and say “that's why”.’ (GP 17)
‘If we know they are unsuitable for whatever reason, then we will exception report that, but we'll also put in the reason why, not just because they are housebound or things like that. It's for proper reasons’. (Practice nurse 23).
Participants recognised the need for a balance between trust and monitoring when using discretionary codes:
‘There are some areas where it needs to be discretionary … There's got to be some degree of trust I suppose within this, and it's very difficult isn't it, to say “how do you make sure that people aren't gaming the system” and, you know, that gaming goes on whatever, and the only way you can counter that is by careful monitoring and having those appropriate discussions.’ (GP 24)
A few participants alluded to the fact that exception reporting was used inappropriately in other practices they knew, or had heard, of:
‘I know some practices have milked it to boost their QOF monies.’ (PM 14)
Two participants in different practices admitted to very occasional inappropriate exception reporting, but still maintained that their practices had low exception reporting rates overall:
‘We try and stick to the rules, I think occasionally people get exception reported for reasons that, perhaps, they shouldn't be, but we have very low rates of exception reporting.’ (GP 12)
Others referred to the fact that the practice was exceeding the upper threshold for an indicator, and therefore their (low) exception reporting rates made no difference to practice achievement or payment targets:
‘… recently we've been looking at hypertensives and looking at whether they're meeting the QOF. And because we've looked at our figures and we've seen that we have actually met the targets, we then haven't gone through the hypertensives and identified anybody who's on maximum treatment because we haven't needed to, because we already hit the targets anyway.’ (GP 6)
A few GPs spoke about how patients were considered for exception reporting as practice staff saw them throughout the year (whether at review or opportunistically), but most described a greater focus on exception reporting towards the end of the QOF financial year. For some, this was described as good practice in trying to make a clinical register as complete as possible but, for others, it was also an opportunity to see whether targets had been met and, if not, whether exception reporting should be considered for remaining patients to hit the target. As one practice manager explained:
‘I think it's probably much more, you know, nearer the end of the year when you realise that you're struggling to make this target that you may actually then go back and say, well, why have we not done this one, this one, and look at and see whether it's reasonable that they've not done exception coding that's relevant.’ (PM 31)
The threat of external scrutiny on behaviour
A majority of participants referred to how their declared minimal use of exception reporting was due, in part, to a perception that exception reporting rates were ‘monitored’, ‘policed’, and ‘scrutinised’ by their PCT. The thought that their PCT QOF team may check up and challenge them on their amount of exception reporting, led some to emphasise and acknowledge the need for a reasonably low level of exception coding and to justify each discretionary exception code:
‘There's an idea that if you exception report you'll then get your system interrogated and you might have picked up someone spying on you, whatever it is. So you try to keep them [exception reporting] to a minimum.’ (GP 18)
‘The problem with exception coding is that we are being criticised by a PCT. So you can't win either way really. And if you exception code on legitimate grounds then it seems unfair to have your reputation for performance tarnished by an allegation of exceptional exception reporting. So it does place you in a really difficult spot.’ (GP 29)
However there was also a tension, recognised by a majority of participants that, although exception reporting rates should be low ethically and clinically, not exception coding could produce effects that were equally negative. Just missing a threshold to achieve maximum points would have a negative effect on practice income and, therefore, also on subsequent patient care:
‘I mean, that's the dilemma, isn't it? Because, as you say, you can exception code but there is then that concern for the practices that, if they're seen to be exception coding out with the norm, they could be penalised and there are financial constraints on all practices that mean they are having to try and keep their incomes to a maximum, because it will affect services if they don't.’ (GP 31)
DISCUSSION
Summary of the main findings
This study suggests that exception reporting is seen by practice staff as an important and clinically necessary part of the QOF. Exempting patients was generally seen as an ‘exception to the rule,’ and inappropriate exempting was largely done by ‘other’ practices. When exception reporting was used, it was justified either in terms of practising patient-centred care within a framework of population-based health measures, or because of the poor face validity of the indicators themselves for individual patients.
The threat of external scrutiny from PCTs was recognised as a driver to keep rates low. However GPs were happy to – and believed it was legitimate to have to – defend the use of discretionary exception codes for individual patients. Some participants also acknowledged that exception reporting was used, particularly at the end of the financial year, to help meet unmet targets and to prevent the practice being penalised financially. Overt use as a means of ‘gaming’ was mentioned by GPs in only two of the 27 participating practices.
Strengths and limitations of the study
To the authors' knowledge, there have been no previous qualitative studies of general practice staff views on exception reporting. Interviewees were from a nationally representative sample of general practices and were those most involved in piloting and general day-to-day management of the QOF.
Exception reporting a patient due to clinical judgment is based on the GP's views and not the patient's – as such, no patients were interviewed. However, the clinical basis for excluding patients was not corroborated with data from their medical records.
Comparison with existing literature
Weick termed the central activity in all organisations as ‘sensemaking’ – ‘an ongoing accomplishment that takes form when people make retrospective sense of the situations in which they find themselves and their creations.’20 In a health context, it has been used to make sense of reflective thinking in nursing practice,21 the implementation of the new general medical services contract,22 and practice-based commissioning.23 Sensemaking is an ongoing cycle of reaction to the world and action that changes that world, driven by internal factors such as previous experiences, beliefs, and values.20 The way members of a practice team ‘make sense’ of the idea of exception reporting will not only determine how they act internally, but will also influence how they see the world (that is, other practices and the PCT) around them.
The process of sensemaking is rooted in identity construction, and explicit efforts at sensemaking tend to occur when the current state of the world is perceived to be different from the expected one. To make sense of disruption, people look for reasons that enable them to resume the activity and to stay in action.24 Using sensemaking as a conceptual framework in the context of this study allows us to see the practices' stories as a retrospective attempt to justify their relatively minimal use of exception reporting at a time of uncertainty within the wider NHS. Most GPs in this study saw themselves as patient advocates and did not describe and justify their use of exception reporting as if they were financially driven entrepreneurs. The majority of practice staff made sense of exception reporting as a means of protecting patients rather than protecting income – gaming was seen very much as something done by others.
Any assessment of quality of health care must take into account all the complexities of primary health care,5 including GPs' wishes to espouse and practice holism23 and act as patient advocates. Previous work has recognised a concern that the financial pressure to meet QOF targets may mean there is a risk that the patient's agenda is ignored at the expense of a more biomedical type of care.7–8,15 The nature of patient-centred care requires both patients' and doctors' differing priorities to be negotiated to achieve an ideal outcome.25–26
Implications for clinical practice
This study suggests that discretionary exception reporting is used by GPs when, in their clinical judgment, the agenda of addressing the care or treatment within an indicator is subsumed by the imperative of addressing the needs of the patient. It is an important safeguard against inappropriate or over-treatment of patients. This judgment reflects an active decision to focus on individually tailored care, rather than focusing on generalised population-based care.27
It has been argued previously that it would be inappropriate to replace an indicator for which a large proportion of patients with the condition are excepted, without knowing why this is being done.28 The current study suggests that exception reporting is being used in a suitable manner and that its use within a decision-making system for indicator removal is safe and appropriate; this is reflected in the variation in exception reporting at indicator level (0.5–25%).6
It would appear that most practices value and use exception reporting as a clinical safeguard to quality individual patient care within an evidence-based but largely population-level and inflexible framework.
Acknowledgments
The authors wish to thank the staff in all participating pilot practices, as well as Annette Barber and Evangelos Kontopantelis.
Notes
Funding body
This study was supported by a grant from the Department of Health.
Ethical approval
The study had full ethical committee approval from North West 4 Research Ethics Committee (09/H1001/74).
Competing interests
Stephen Campbell and Helen Lester are contracted to the National Institute for Health and Clinical Excellence (NICE) to provide advice on removal of indicators and pilot new indicators for the Quality and Outcomes Framework. The views expressed are those of the authors and do not necessarily represent the views of NICE or its independent Quality and Outcomes Framework advisory committee.
Discuss this article
Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss
- Received September 24, 2010.
- Revision received October 19, 2010.
- Accepted December 2, 2010.
- © British Journal of General Practice, April 2011