A major part of the new General Medical Services contract implemented across the UK from April 2004 is the Quality and Outcomes Framework (QOF), under which up to a quarter of the general practitioners’ income depends on practice performance measured against 146 clinical and organisational indicators.¹ Indicators are assigned a fixed number of points, which weight them according to perceived workload and importance, each point being worth £75 to an average practice in 2004–5. A total of 76 indicators and 550 of the 1050 points relate to the management of 10 chronic diseases. The indicators capture the maintenance of a disease register, recording of risk factors, recording and achievement of intermediate outcomes, and delivery of recommended treatments. Payment is triggered once performance on each clinical indicator exceeds 25% and increases linearly until performance exceeds 90% for most process measures and 50–70% for most intermediate outcome measures.

Much of the early coverage of the QOF has been on the high average number of points achieved, with widespread coverage at both national and regional levels, indicating that quality of care is generally high, with relatively little variation between practices.² However, two caveats apply to this conclusion. Firstly, maximum points can be achieved for percentages of the target population ranging from 50% to 90% depending on the indicator. Examining quality in terms of the actual percentages of patients achieving particular targets may therefore show variations that are not apparent in the distribution of points. Secondly, practices are allowed to exclude from the measurement of achievement those patients who are unsuitable for any chronic disease management or for particular indicators. Box 1 lists the acceptable reasons for exclusion.^3,4 The intention is to avoid financially penalising practices for the characteristics of the population they serve. However, these reasons to exclude patients are likely to be more common in deprived populations (eg, through less response to invitations to attend clinics on chronic disease management). Measuring quality using payment percentages that account for exclusions may therefore conceal continuing inequalities in service provision^5–8 and perpetuate them by failing to reward practices for the additional work required to bring deprived populations into systematic care.

Surprisingly, there has been little focus on the possible effect of allowing practices to exclude patients. This paper therefore examines how the quality of primary care varies by levels of deprivation, comparing two measures of the quality of care. “Payment quality” is the percentage of non-excluded patients who achieve a particular target (eg, have their blood pressure measured, have their cholesterol controlled or have an influenza vaccination). This is the measure that determines payment under the QOF. “Delivered quality” is the percentage of all patients with the disease who achieve a particular target (box 2). This informs consideration of whether the QOF has helped exclude longstanding inequalities in care.^5,6

METHODS

Using publicly available data on QOF achievement for each practice,⁹ we calculated payment quality and delivered quality for clinical indicators, where the true denominator was all patients with the disease (box 2). Table 1 shows the indicators used for calculating payment quality and delivered quality.

Numerators and denominators for payment were measured on 31 March 2005. As prevalence determines payment level (pounds per point), the number of patients on the disease register was recorded by the Quality Management and Assessment System (QMAS).¹⁰ For most practices, these prevalence levels were extracted on 14 February 2005, “National Prevalence Day”. Initial analysis indicated that in some practices, the reported register size on 14 February was smaller than the denominators for each indicator on 31 March, indicating that it is unreliable. The two may differ because

• individuals with a diagnosis leave the practice or die;

• new individuals with a diagnosis arrive at the practice;

• new cases are diagnosed between 14 February and 31 March; and

• cases diagnosed on 14 February are found inaccurate and deleted by 31 March.

We therefore estimated the register size on 31 March using the maximum value of any denominator in the relevant clinical domain. This assumes that each practice has not excluded anyone from at least one of its indicators. It will therefore underestimate the true register size on 31 March and overestimate delivered quality. For the purpose of this analysis, the key issue is whether the estimate is systematically related to deprivation. No systematic relationship was found between deprivation and differences between the reported register size on 14 February and our estimate of register size on 31 March, which indicates that our results are unlikely to be due to bias in this method for estimating the total register size.

Our measure of practice deprivation is derived from the income domain of the Scottish Index of Multiple Deprivation, 2004.¹¹ We chose the income domain rather than the overall score or other domains, because it receives the highest weight in the calculation of the overall index and is highly correlated with it, is available for small geographical areas representing between 500 and 1000 people and does not undergo transformations, making it easily interpretable. Average values for practices were calculated on the basis of the geographical distribution of registered populations.

QOF data were available for 1024 practices, although not all practices participated in each of the clinical domains examined, and one practice was excluded because no deprivation data were available. We provide results for payment quality (based on payment denominators) and delivered quality (based on total register size) for indicators where the true denominator is for all patients with the disease, for five clinical areas—coronary heart disease (CHD), stroke and transient ischaemic attack, diabetes mellitus, hypertension and chronic obstructive pulmonary disease (COPD; table 1). The chosen indicators represent 294 (53%) of the 550 points available for clinical quality.

We estimated the size and statistical significance of the trend across deprivation using regression analysis. We used linear regression throughout, estimated using Stata V.8 with robust standard errors. The calculation of mean values and the regression coefficients were weighted by population size.

The regression coefficients represented the change in quality associated with a one-point increase in the percentage of the practice population receiving state benefits on the grounds of low income. We compared the regression coefficients when quality was measured by payment quality and delivered quality. Where the coefficient on delivered quality minus the coefficient on payment quality is negative, the implied exclusion rates are higher in more deprived practices. Exclusion rates are lower in deprived practices if the coefficient on delivered quality minus the coefficient on payment quality is positive.

RESULTS

Table 1 gives the definitions, payment stages and national mean values for both payment quality and delivered quality. Absolute differences between payment quality and delivered quality for simpler process measures such as recording of smoking status, blood pressure, cholesterol and creatinine are generally small. Larger differences are found for indicators for more complex processes such as foot and eye screening in diabetes, and spirometry in COPD, and for intermediate outcomes. The largest differences are found for treatment indicators with β-blockers falling 17.1% and the three immunisation indicators for CHD, stroke and diabetes falling 10.8%, 12.1% and 13.3%, respectively.

On the process indicators (table 2), payment quality is significantly (p<0.05) higher in more affluent practices for two of the 18 indicators, significantly (p<0.05) higher in more deprived practices for seven indicators and not significantly different for the remaining nine indicators. By contrast, delivered quality of care is better in more affluent practices on eight indicators, better in more deprived practices on one indicator and not significantly different on the remaining nine indicators. The most dramatic differences are for the two diagnostic indicators for COPD. For the monitoring of patients with COPD using forced expiratory volume in 1 s (FEV₁), the regression coefficient changes from 0.35 for payment quality to −0.45 for delivered quality. The range in deprivation is 40.89 points, so this equates to a maximum change in measured quality of 32.7% (payment quality being 14.3% higher in the most deprived practices, whereas delivered quality is 18.4% lower). Three complex process diabetes measures (vascular and neuropathic foot screening, and eye screening) show similar reversals.

On the intermediate outcome indicators (table 3), payment quality is significantly higher in more affluent practices for one of the nine indicators and higher in more deprived practices on four indicators. Delivered quality is higher in more affluent practices on four indicators, and higher in more deprived practices on three indicators (although for two of these, the relationship is weakened). Only cholesterol control in patients with CHD shows a stronger negative relationship with deprivation on payment quality than with delivered quality. Three indicators have a positive relationship with deprivation on both payment and delivered quality. The indicators reflecting blood pressure control for patients with hypertension and HbA1c levels <10% for patients with diabetes both become more negatively and significantly related to deprivation under delivered quality. The indicator reflecting HbA1c levels <7.4% for patients with diabetes is the only outcome indicator that moves from having a significant positive relationship with deprivation on payment quality to a significant and negative relationship with deprivation on delivered quality.

On the treatment indicators (table 4), payment quality is significantly higher in more affluent practices for two of the six indicators and higher in more deprived practices for just one indicator. Delivered quality is higher in more affluent practices for five indicators. None of the indicators show a significant positive relationship between deprivation and delivered quality. Only the taking of aspirin or equivalent drugs for patients with CHD shows a significant negative relationship between deprivation and delivered quality. Both influenza vaccination indicators for diabetes and COPD move from having a significant positive relationship with payment quality to a significant and negative relationship with delivered quality. The taking of β-blockers for patients with CHD becomes more negatively and significantly related to deprivation on delivered quality and influenza immunisation of patients with CHD becomes more negatively related to deprivation and remains significant.

DISCUSSION

The introduction of the new General Medical Services reflects international interest in using financial incentives as a method of improving primary care. As such, the results of the new UK contract will attract much interest.¹²

This is the first study to compare quality using the measures that drive payment with quality measured in terms of the actual care delivered. Its major limitation reflects the way that data are held in the payment system. Delivered quality can only be estimated for a limited number of indicators because QMAS does not record either the true denominators for every indicator or the register size on the same date that the indicator data are extracted. The method we have used overestimates delivered quality but is not biased in terms of estimating the effect of deprivation.

At face value, new contract data for Scotland indicate that there is little systematic difference between practices serving populations with different levels of deprivation. It is consistent with financial incentives for quality such as the QOF mitigating previous inequalities in care.^5–8,13,14 However, such conclusions based on payment quality are misleading. For 17 of the 33 indicators, delivered quality is systematically lower in practices serving more deprived populations. It is higher for only four indicators. This indicates that the exclusions system of the QOF conceals continuing inequalities in service provision. The absolute differences are relatively small for simpler process measures that can be delivered opportunistically (measurement of blood pressure and cholesterol and recording of smoking status), but are considerably larger for many diagnostic, intermediate outcome and treatment indicators.

There are two main explanations for these patterns. Firstly, more deprived populations may have more legitimate reasons for exclusion, such as not attending routine care, having greater comorbidity, and moving and therefore changing practice more often. Secondly, practices serving more deprived populations may be of lower quality generally and use exclusions to maximise payment. This is facilitated by the relative lack of definition of exclusions compared with the clarity of the indicators themselves.

Identifying which of these explanations is true will require more detailed examination than is possible in this study. The first explanation is plausible, but many reasons for exclusion that are legitimate under the contract (such as poor attendance at routine clinics) are potentially amenable to health service intervention.^15,16 Moreover, if the problem was a result of greater “no show” rates among the disadvantaged, it is hard to see why the implied exclusion rates differ across the same indicator for different diseases.

The payment system can be considered to have been successful in not penalising practices financially for the populations that they serve. However, it highlights that the contract does not reward for the extra work involved in delivering equal outcomes and treatments to disadvantaged populations. Furthermore, the indicators focus on levels of outcome, rather than changes in outcome. As some patients may achieve the outcome indicators without intervention, particularly in more affluent areas, practices may be rewarded for levels of outcome that are not attributable to their services.

This study also highlights the limitations of QMAS data in examining the distribution of quality of care using different assumptions about exclusions. Notably, many of the diagnostic and treatment indicators do not have a recoverable true denominator (eg, most of the diagnostic indicators and some treatment indicators are limited to new diagnoses and the smoking advice indicator is based on the denominator of current smokers). As inequalities are greatest for similar indicators where we can recover a true denominator, the results presented here are likely to underestimate the extent of inequalities in delivered care.

In conclusion, this paper has shown that despite payment data indicating a relatively equitable distribution of quality under the 2004 general practice contract, the inverse care law still applies with persistent inequalities in the care actually delivered.^5,6 Analysis of this inequality is hampered by the lack of information about exclusions in QMAS in 2004–5. The inclusion of data on exclusions in QMAS for the 2005–6 contract year will allow a more accurate analysis of inequalities in care across socioeconomic groups. Only then will the full effect of these financial incentives on equity be understood, and practices and primary care organisations be informed to make an appropriate response. Meanwhile, the lack of information on the reasons for exclusion limits the lessons that can be learned in other countries about the effect of this innovative set of incentives in improving the equity and quality of care.