Exception reporting in the Quality and Outcomes Framework: views of practice staff – a qualitative study

Sample and study participants

Practices were recruited as part of a wider ongoing process that involves piloting potential quality indicators for the QOF.1 Piloting involves the recruitment of separate cohorts of representative practices in 12 geographically disparate primary care trust (PCT) areas of England, and two practices each in Northern Ireland, Scotland, and Wales. Participants for this study were recruited from 28 practices from the first cohort of pilot practices (October 2009–March 2010). This English pilot cohort included practices that were nationally representative in terms of size, QOF achievement, Index of Multiple Deprivation (Table 1), and the use of different IT systems.

Participants were nominated from within those practices taking part but, to be eligible for interview in the study, had to have played a leading role in the practice's day-to-day engagement with the QOF. Sixty-six general practice professionals and staff across 27 of the pilot practices were interviewed; interviews lasted around 45 minutes. The participants consisted of 24 GPs (from 23 practices), 20 practice managers (from 20 practices), and 13 nurses (from 11 practices). Nine other staff (from eight practices) who were, for the most part, IT staff or medical notes summarisers were also interviewed. Seven of the GPs were female and 17 were male; all were principal partners. Two-thirds of the GPs worked full time. All nurses were female, with two-thirds working part time.

Data collection

Semi-structured interviews were conducted in the participants' own general practice workplaces between April and May 2010. An interview topic guide, based on the objectives of the study, was used (Box 2).

Data analysis

All interviews were digitally recorded and fully transcribed. The study was explorative, not theoretical, so open, rather than axial or selective, coding was used.16 Each transcript was read and coded separately by two of the authors and a preliminary inductive coding frame constructed.

Transcripts were coded inductively without knowing what would be found when the data analysis began. Upon completion of coding, those codes that had common elements were merged to form categories; this was to ensure that the analysis of the transcripts reflected the recurring and representative themes found.17 Disconfirming evidence was actively sought throughout.18

The findings are based on a synthesis of all the interviews in this study. All statements are based on the reports of multiple interviewees. The quotations were chosen on the grounds of representativeness.

Reasons for exception reporting

All participants were asked whether their practice had any formal policy or underlying ground rules for exception reporting eligible patients from the QOF. No practice had a formal, written practice policy on exception reporting, but most focused on two key principles. The first of these related to adherence to the external national guidance rules on (non-discretionary) exception reporting19 as well as a parallel, internal, practice-specific focus reflecting on the suitability of individual patients for discretionary exclusion.

The ‘externally set’ rules particularly related to non-discretionary criteria, such as patients not responding to or refusing three invitations to attend a review and those not agreeing to an investigation or treatment (informed dissent):

‘We don't exception report unless we can avoid it. It's very much [a] last resort. If a patient clearly doesn't respond to our efforts to get them in, then they will be exception coded.’ (practice manager [PM] 6)

‘… if a patient has been written to on at least three occasions, and the patient does not attend at that point, then we can consider exception reporting. Secondly, if a patient clearly indicates in person they do not wish to be involved, we will exempt the patient.’ (PM 8)

These exception codes were often entered onto to the practice system by employed non-clinical practice staff.

Given that not all indicators were equally relevant to the needs of each individual patient on a given register, the second underlying principle related to an assertion that exception reporting was an integral part of the clinical judgment required in implementing indicators. This use of clinical judgment was expressed in two ways, one of which was through the use of specific ‘discretionary’ exception rules, such as patients being on maximal tolerated therapy. Participants also spoke of how exception reporting allowed them to focus on an individual patient's circumstances, contexts, or choices; for example, where a patient was old and frail or had multiple morbidities. This use of discretionary clinical judgment exception codes was always decided on, and almost always entered, by GPs.

A significant minority of GPs also stated that they were more likely to exception report indicators with poorly perceived face validity. This was because the care/treatment element of the indicator was felt to be contrary to their role as the patient's advocate and, in their clinical judgement, not relevant to individual patient-centred care:

‘I think you do need scope to exception report patients because people are really quite individual and unique, and I think, you know, in the end there is always an element of clinical judgment.’ (GP 29)

‘There is a place for exception reporting within [the] QOF. Discretionary versus non-discretionary, I think you probably have to keep both because of the complexities of the whole agenda. There are some areas where it needs to be discretionary … There are patients out there that just do not fit into this very nice, structured approach.’ (GP 24)

Clinical examples were primarily focused on indicators for depression and chronic renal failure:

‘The depression indicators with the PHQ [Patient Health Questionnaire], I think there's quite a strong feeling among the profession that that's been quite counter-productive to patient care … I think it's important that it enhances care for patients rather than detracts from patient care and these things we're doing actually reflect that. So it's not just a paper exercise, there is actually something that's going to add to quality really’. (GP 6)

A significant minority of participants referred to societal issues – such as the effect of deprivation on patient compliance, health-seeking behaviour, and morbidity – that related to particular groups of patients in their practice:

‘If you've got practices in very deprived areas they're going to have a lot more patients who won't attend …’ (GP 13)

Patients with, for example, serious mental illness were described as often consulting opportunistically, rather than at specific appointment times:

‘But mental health are particularly difficult because they're not the sort of patients you can say “come back next week”.’ (GP 19)

One family doctor spoke specifically about how unfair it would be to remove exception reporting from practices in areas of deprivation, and another made the same point with regard to patients who often don't attend when specifically invited to.

The level and appropriateness of exception reporting

Most participants emphasised that their own practice had low rates of exception reporting and said it was ‘not used lightly’ (PM 24) or that it was used ‘minimally’ (PM 26). Other responders stated that it was used ‘conservatively’ (GP 18), ‘ only when necessary’ (PM 17), or as ‘very much [a] last resort’ (PM 6) and in ‘extreme cases’ (PM 3). They felt that each and any use had to be defensible:

‘The only rule I think we have is that we feel that every exempt reporting needs to be defendable. So if you ask me why did you exempt this patient I should have an answer … even if you ask me 3 years later I should be able to look at a note and say “that's why”.’ (GP 17)

‘If we know they are unsuitable for whatever reason, then we will exception report that, but we'll also put in the reason why, not just because they are housebound or things like that. It's for proper reasons’. (Practice nurse 23).

Participants recognised the need for a balance between trust and monitoring when using discretionary codes:

‘There are some areas where it needs to be discretionary … There's got to be some degree of trust I suppose within this, and it's very difficult isn't it, to say “how do you make sure that people aren't gaming the system” and, you know, that gaming goes on whatever, and the only way you can counter that is by careful monitoring and having those appropriate discussions.’ (GP 24)

A few participants alluded to the fact that exception reporting was used inappropriately in other practices they knew, or had heard, of:

‘I know some practices have milked it to boost their QOF monies.’ (PM 14)

Two participants in different practices admitted to very occasional inappropriate exception reporting, but still maintained that their practices had low exception reporting rates overall:

‘We try and stick to the rules, I think occasionally people get exception reported for reasons that, perhaps, they shouldn't be, but we have very low rates of exception reporting.’ (GP 12)

Others referred to the fact that the practice was exceeding the upper threshold for an indicator, and therefore their (low) exception reporting rates made no difference to practice achievement or payment targets:

‘… recently we've been looking at hypertensives and looking at whether they're meeting the QOF. And because we've looked at our figures and we've seen that we have actually met the targets, we then haven't gone through the hypertensives and identified anybody who's on maximum treatment because we haven't needed to, because we already hit the targets anyway.’ (GP 6)

A few GPs spoke about how patients were considered for exception reporting as practice staff saw them throughout the year (whether at review or opportunistically), but most described a greater focus on exception reporting towards the end of the QOF financial year. For some, this was described as good practice in trying to make a clinical register as complete as possible but, for others, it was also an opportunity to see whether targets had been met and, if not, whether exception reporting should be considered for remaining patients to hit the target. As one practice manager explained:

‘I think it's probably much more, you know, nearer the end of the year when you realise that you're struggling to make this target that you may actually then go back and say, well, why have we not done this one, this one, and look at and see whether it's reasonable that they've not done exception coding that's relevant.’ (PM 31)

The threat of external scrutiny on behaviour

A majority of participants referred to how their declared minimal use of exception reporting was due, in part, to a perception that exception reporting rates were ‘monitored’, ‘policed’, and ‘scrutinised’ by their PCT. The thought that their PCT QOF team may check up and challenge them on their amount of exception reporting, led some to emphasise and acknowledge the need for a reasonably low level of exception coding and to justify each discretionary exception code:

‘There's an idea that if you exception report you'll then get your system interrogated and you might have picked up someone spying on you, whatever it is. So you try to keep them [exception reporting] to a minimum.’ (GP 18)

‘The problem with exception coding is that we are being criticised by a PCT. So you can't win either way really. And if you exception code on legitimate grounds then it seems unfair to have your reputation for performance tarnished by an allegation of exceptional exception reporting. So it does place you in a really difficult spot.’ (GP 29)

However there was also a tension, recognised by a majority of participants that, although exception reporting rates should be low ethically and clinically, not exception coding could produce effects that were equally negative. Just missing a threshold to achieve maximum points would have a negative effect on practice income and, therefore, also on subsequent patient care:

‘I mean, that's the dilemma, isn't it? Because, as you say, you can exception code but there is then that concern for the practices that, if they're seen to be exception coding out with the norm, they could be penalised and there are financial constraints on all practices that mean they are having to try and keep their incomes to a maximum, because it will affect services if they don't.’ (GP 31)

Summary of the main findings

This study suggests that exception reporting is seen by practice staff as an important and clinically necessary part of the QOF. Exempting patients was generally seen as an ‘exception to the rule,’ and inappropriate exempting was largely done by ‘other’ practices. When exception reporting was used, it was justified either in terms of practising patient-centred care within a framework of population-based health measures, or because of the poor face validity of the indicators themselves for individual patients.

The threat of external scrutiny from PCTs was recognised as a driver to keep rates low. However GPs were happy to – and believed it was legitimate to have to – defend the use of discretionary exception codes for individual patients. Some participants also acknowledged that exception reporting was used, particularly at the end of the financial year, to help meet unmet targets and to prevent the practice being penalised financially. Overt use as a means of ‘gaming’ was mentioned by GPs in only two of the 27 participating practices.

Strengths and limitations of the study

To the authors' knowledge, there have been no previous qualitative studies of general practice staff views on exception reporting. Interviewees were from a nationally representative sample of general practices and were those most involved in piloting and general day-to-day management of the QOF.

Exception reporting a patient due to clinical judgment is based on the GP's views and not the patient's – as such, no patients were interviewed. However, the clinical basis for excluding patients was not corroborated with data from their medical records.

Comparison with existing literature

Weick termed the central activity in all organisations as ‘sensemaking’ – ‘an ongoing accomplishment that takes form when people make retrospective sense of the situations in which they find themselves and their creations.’20 In a health context, it has been used to make sense of reflective thinking in nursing practice,21 the implementation of the new general medical services contract,22 and practice-based commissioning.23 Sensemaking is an ongoing cycle of reaction to the world and action that changes that world, driven by internal factors such as previous experiences, beliefs, and values.20 The way members of a practice team ‘make sense’ of the idea of exception reporting will not only determine how they act internally, but will also influence how they see the world (that is, other practices and the PCT) around them.

The process of sensemaking is rooted in identity construction, and explicit efforts at sensemaking tend to occur when the current state of the world is perceived to be different from the expected one. To make sense of disruption, people look for reasons that enable them to resume the activity and to stay in action.24 Using sensemaking as a conceptual framework in the context of this study allows us to see the practices' stories as a retrospective attempt to justify their relatively minimal use of exception reporting at a time of uncertainty within the wider NHS. Most GPs in this study saw themselves as patient advocates and did not describe and justify their use of exception reporting as if they were financially driven entrepreneurs. The majority of practice staff made sense of exception reporting as a means of protecting patients rather than protecting income – gaming was seen very much as something done by others.

Any assessment of quality of health care must take into account all the complexities of primary health care,5 including GPs' wishes to espouse and practice holism23 and act as patient advocates. Previous work has recognised a concern that the financial pressure to meet QOF targets may mean there is a risk that the patient's agenda is ignored at the expense of a more biomedical type of care.7–8,15 The nature of patient-centred care requires both patients' and doctors' differing priorities to be negotiated to achieve an ideal outcome.25–26

Implications for clinical practice

This study suggests that discretionary exception reporting is used by GPs when, in their clinical judgment, the agenda of addressing the care or treatment within an indicator is subsumed by the imperative of addressing the needs of the patient. It is an important safeguard against inappropriate or over-treatment of patients. This judgment reflects an active decision to focus on individually tailored care, rather than focusing on generalised population-based care.27

It has been argued previously that it would be inappropriate to replace an indicator for which a large proportion of patients with the condition are excepted, without knowing why this is being done.28 The current study suggests that exception reporting is being used in a suitable manner and that its use within a decision-making system for indicator removal is safe and appropriate; this is reflected in the variation in exception reporting at indicator level (0.5–25%).6

It would appear that most practices value and use exception reporting as a clinical safeguard to quality individual patient care within an evidence-based but largely population-level and inflexible framework.