Abstract
Background The ideal population size of healthcare commissioning organisations is not known.
Aim To investigate whether there is a relationship between the size of commissioning organisations and how well they perform on a range of performance measures.
Design and setting Cross-sectional, observational study of performance in all 152 primary care trusts (PCTs) in England.
Method Comparison of PCT size against 36 indicators of commissioning performance, including measures of clinical and preventative effectiveness, patient centredness, access, cost, financial ability, and engagement.
Results Fourteen of the 36 indicators have an unadjusted relationship (P<0.05) with size of the PCT. With 10 indicators, there was increasing quality with larger size. However, when population factors including deprivation, ethnicity, rurality, and age were included in the analysis, there was no relationship between size and performance for any measure.
Conclusion There is no evidence to suggest that there is an optimum size for PCT performance. Observed variations in PCT performance with size were explained by the characteristics of the populations they served. These findings suggest that configuration of clinical commissioning groups should be geared towards producing organisations that can function effectively across their key responsibilities, rather than being based on the size of their population alone.
- efficiency, organizational
- health facility size
- health services research
- primary health care
- organization & administration
- quality indicators
- health care
- statistics & numerical data
INTRODUCTION
The government’s reforms of the NHS in England have been through a number of iterations.1-3 The latest proposal, after the government’s ‘pause’ and ‘listening exercise’, looks set to develop clinical commissioning groups (CCGs) with responsibility for commissioning local services and controlling budgets. These will combine local GPs with the input of secondary care clinicians and others.3 The size, structure, and exact function of these groups remain unclear. The question of what size commissioning organisations should be to allow them to function most effectively is therefore highly relevant for local and national decision makers.4
Several hypotheses about the relationship between size of commissioning organisation and performance exist.5 These include the concern that small commissioning units are more exposed to financial risk, due to their smaller populations, and consequently their smaller risk pool. Similarly, smaller commissioning units may not have a critical mass of expertise or the required ‘market power’ to be able to negotiate effectively with providers to achieve good-value contracts. Alternatively, a smaller commissioning unit size may allow better local engagement and responsiveness for clinicians and patients.
The ongoing lack of consensus about the optimal size of commissioning units, leading to consequent reorganisations, has proved highly disruptive and costly to healthcare systems. The history of the NHS contains several reorganisations of the commissioning function, since the introduction of the purchaser–provider split in the early 1990s. Over this period, organisational units of different size have been tried. In April 1999, there were 481 primary care groups, with an average population of around 100 000.6 In 2005, there were 302 primary care trusts (PCTs, average population 170 000).7 Since 2010, there are now 152 PCTs (average population 290 000).8 The question of ideal organisational size is not limited to the UK health system, as other countries have experimented with different-sized organisational models. In Australia, New South Wales is currently reforming the size of organisational units from area health services (serving populations of around 1 million) to local health networks (serving 500 000),9,10 having reformed the size of the units only 5 years previously.11 In the US, there is a continuing discussion about the optimal size of physician groups and networks to provide the highest-quality care.12
The relationship between size of commissioning units and organisational performance remains under-investigated. A previous analysis of primary care organisations in England in 2003 examined a random sample of 72 commissioning organisations, and their performance against 22 measures.13 Using a mixture of telephone interviews and survey questionnaires, it was found that only two performance measures — provision of extended services and provision of intermediate care — were significantly associated with size. However, national analyses are lacking, and, since this study was carried out, performance measurement in the NHS has improved, new data are available at national level, variation between organisations may have reduced, and the optimum size of CCGs as the new commissioning organisations in the UK, remains contested. These factors all suggest it is appropriate to look at the question again.
How this fits in
Commissioning organisations vary in size and performance. New commissioning organisations will be created in the latest NHS reforms. Previous studies have found little evidence of a link between size and performance, but no national analysis has been done before. This study shows a link between commissioning organisation size and some areas of performance, but this is explained by the characteristics of the population. There is still no evidence of an ideal size for clinical commissioning groups in the new NHS reforms.
METHOD
Selecting measures of performance
Measuring the performance of commissioning organisations is difficult, in part because they have many different functions. Several frameworks for measuring quality of health care exist, with differing but often overlapping definitions. In order to ensure that this evaluation of performance considers the various aspects, the Organisation for Economic Cooperation and Development’s (OECD’s) framework for healthcare system performance was used,14 which focuses on healthcare quality and divides performance into five separate domains: effectiveness, safety, patient centredness, access, and cost. Two additional domains were added: ability to engage with the public and the local health economy and financial ability, as these are also stated aims of PCTs.15
Thirty-six established and commonly used performance measures across these domains were selected (Table 1). Measures of effectiveness were broken down into clinical and preventative effectiveness, reflecting the role of commissioning organisations in both of these areas. No performance measures were included for safety, as it was not possible to identify any that had sufficient standardisation and availability across all PCTs.
Data collection
Data were obtained on each performance indicator for all 152 PCTS in England. The most recent data available were extracted from a number of different data sources (Table 1). Sources included the National Centre for Health Outcomes Development,33 the Quality and Outcomes Framework (QOF; a national payment incentive framework for general practice),34 NHS Comparators (a national comparative performance tool run by the NHS),35 World Class Commissioning reports (WCC; a national commissioning performance assessment exercise),23 and the national General Practice Patient Survey.28 For almost all performance indicators, data were publicly available on the internet, and obtained via this route. Where data were missing on WCC performance,31 contact was made with five individual PCTs to provide their WCC assessments, and all provided this information. Data on PCT size were taken from the NHS Information Centre, 2009–2010, and counted the registered population in each area.8
Performance may be affected by a number of confounding variables, apart from size, that relate to the populations PCTs serve. For each PCT, information was therefore obtained on age structure (% >65 years),36 ethnicity (% white),37 level of deprivation (Index of Multiple Deprivation [IMD]),38 and rurality (measured as a binary variable: urban, corresponding to predominantly urban in the Office for National Statistics [ONS] rurality classification; and ‘rural’, corresponding to the ‘predominately rural’ and ‘significantly rural categories’).39 PCTs receive additional funding to compensate for these factors as well as for population size.
Data analysis
Population size was compared against each of the performance indicators at the PCT level, using Spearman’s rank correlation as a non-parametric measure of statistical dependence between two variables.
To look for potentially confounding relationships on indicators with a significant correlation coefficient, a univariate linear regression analysis (using ordinary least squares) was performed, comparing each performance indicator with each of: PCT size, level of deprivation, ethnicity, age, and rurality. To investigate the relationship between size, performance, and other potentially confounding variables, a multiple linear regression model was used, which included deprivation, age, and ethnicity as continuous variables and rurality (rural or urban) as a categorical variable, allowing the relative contributions of each variable to be understood. Data analysis was performed with STATA SE (version 11).
RESULTS
Unadjusted findings
Initial analysis of PCT size against the performance indicators shows that 14 of the 36 indicators had a significant correlation (P<0.05) between performance and population size. For 11 indicators, this was at the P<0.01 level (Table 2).
The commonest relationships between size and performance are for clinical effectiveness (six out of nine indicators) and preventative activity (three out of five indicators). There is less relationship between size and measures of access (two out of five), cost (two out of five), and patient experience (one out of six). There is no observed relationship between size and measures of commissioning ability or financial ability.
The general trend is that bigger PCTs provide better services. However, there are anomalies, for example bigger PCTs also have lower rates of smoking quitting, lower rates of generic statin prescribing, and lower satisfaction with opening hours. There is also a relationship between size of PCT and the average level of deprivation within it (Spearman P<0.001, larger PCTs are less deprived) and between size and rurality (Spearman P<0.001, larger PCTs are more rural). There was no observed relationship between size and ethnicity or age structure using Spearman’s rank test.
Adjusted findings
There were a number of relationships observed between the potentially confounding variables and the performance measures. Considering the 14 variables where there is a relationship with PCT size and performance on the Spearman analysis, there is also a relationship between deprivation and performance for 12 of the 14 indicators at the P<0.01 level, age (eight of 14), ethnicity (eight of 14), and rurality (10 of 14).
In the combined regression model that included population characteristics, PCT size is a much weaker predictor of performance than the other variables (Table 3). Size is no longer a significant contributor to the model for any variable (P<0.05). In contrast, deprivation is significant for 12 of 14 indicators (10 of 14 at P<0.001), ethnicity for 10 of 14 (10 of 14 at P<0.001), age in seven of 14 (six of 14 at P<0.001), and rurality in one of 14 (0 of 14 at P<0.001).
DISCUSSION
Summary
These results suggest that there is a relationship between the size of PCTs and organisational performance in a number of areas of their activity, particularly clinical and preventative effectiveness. However, the relationship is no longer present when population characteristics such as deprivation are taken into account.
Where there is an unadjusted relationship with size, larger PCTs tend to provide higher-quality care (in 10 out of 14 indicators). An explanation might be that larger PCTs do better because they are more likely to serve affluent, rural, less ethnically diverse populations. This may be a result of the recent pattern of reorganisations that has tended to leave smaller PCTs in urban areas (and London in particular), and larger PCTs in rural counties.
Deprivation appears to be the factor that influences performance for the most variables, but ethnicity is also a strong factor in some areas such as screening programme coverage. This is consistent with other work highlighting poor knowledge regarding screening and low uptake of breast and cervical screening programmes in certain ethnic populations.40,41
The fact that a few indicators seem to demonstrate worse health care with increasing size — for example, larger PCTs have lower smoking quit rates, poorer satisfaction with opening hours, and less-efficient prescribing — is an interesting anomaly. The lower rate of smoking quitters may be because rates of smoking are higher in more deprived, smaller, urban PCTs and the denominator for this indicator is total population size rather than total number of smokers. In these areas, extra resources may be directed towards ‘stop smoking’ campaigns. This is reinforced by the finding that deprivation appears to be the most important explanatory factor in the combined model relating to the smoking quit rate. Lower rates of satisfaction with opening hours may reflect actual differences in activity, or may reflect different expectations of different groups within the populations in PCTs.42
Some domains of quality appear more likely to be related to PCT size than others, in particular measures of effectiveness. There was no relationship, even unadjusted, between size and financial performance or engagement for any of these indicators. This suggests that the hypotheses that smaller PCTs are less able to negotiate contracts effectively, are more exposed to financial risk, and are better able to engage with their community and partners are not supported by the evidence at the current size of PCTs.
Strengths and limitations
There are a number of limitations in the methods used to assess performance of commissioning organisations. Although the study attempted to measure PCT performance across a broad remit, no system for measuring quality will be able to capture all aspects of performance. Moreover, a lack of correlation between various measures of PCT quality has been observed before.43
This paper presents a set of measures, based on an established framework (OECD) that cut across a number of different aspects of performance, and relate to evidence-based guidance for prevention and treatment of common clinical conditions. New national data have been used for the first time, and the study has taken advantage of the more vigorous approach to performance management in recent years to obtain data from many sources. The authors accept that some of the indicators used are not validated. In particular, this includes many of the newer indicators relating to financial and engagement performance, as these have only emerged recently from WCC assessments. Also, these indicators have varying levels of accuracy and completeness of data, and some of the study indicators were based on data collected before 2009/2010, and may not reflect current PCT performance.
Furthermore, some of the outcome variables used may be better than others in terms of reflecting PCT influence. For example, performance on QOF and prescribing indicators may be driven by individual GP performance, and mortality rates by population factors, whereas access to primary care and WCC scores may be more likely to be influenced in the short term by the commissioning organisation. However these results show that relationships between size and performance are observed across several different domains and data sources.
There are also other structural factors that could be viewed as confounders, such as the number of GPs or nurses per 1000 patients. These data were not used, as they were thought to be dependent on PCT commissioning activity. The authors also accept that this analysis makes multiple comparisons. However, significance values have been given at both the 0.05 and 0.01 level, and where relationships are demonstrated, most are highly significant. Due to the nature of the data, causal relationships between PCT size and indicators cannot be concluded.
Comparison with existing literature
These findings are consistent with earlier research on the topic, including a smaller survey of PCT performance by Wilkin et al.13 Bojke et al reviewed the UK and international literature in 2001.44 They suggested that the size of primary care organisations is only one of the factors that affect performance. They suggested a framework in which primary care organisation performance is affected by a combination of their aims, tasks, functions, organisational features (including both size and governance), and environmental factors, including demographic mix and socioeconomic characteristics. They also suggest that there is no optimum size for a commissioning organisation, because there are different economies of scale for different functions and because of the variety of functions the organisations perform.
A similar study in another field, which examined the performance of local government functions in authorities of different size suggests that organisational size has an impact on some areas of performance, but not others, including positive, negative, and non-linear relationships.45 The authors describe the relationship between size and performance as a ‘complex mosaic’. The extent of the effect of deprivation and ethnicity in explaining the link between size and performance has also not been demonstrated so strongly in previous work. The present findings mirror those from practice-level studies that found no strong associations between size of general practices and performance measures, and in which population factors were found to have the greatest impact.46,47
Implications for practice and research
As a result of the complex, confounded, and multidirectional results observed, this analysis fails to provide any conclusive answers to the question of what size a commissioning organisation needs to be to perform best. This adds to the existing literature that suggests that there is no obvious optimum size of commissioning organisations. Rather than pursuing an optimum size, those designing a new commissioning system could instead look to other characteristics of the organisations that might affect performance, such as the internal structure, the strength of its networks with other organisations, and the composition and skills of its workforce.
The cost of the current NHS reorganisation in England has been estimated at £2–3 billion.48 The future configuration of CCGs remains uncertain, but doubts have been expressed about the viability and financial stability of smaller groups. The present analysis suggests that smaller commissioning organisations can function as effectively as larger ones, across a broad range of performance measurement. But, given the enormous pressure to reduce management costs, it is likely that larger CCGs will be the norm. The configuration of CCGs, and similar entities in other health systems, should therefore be geared towards producing organisations that can function effectively across their key responsibilities, rather than just being based on the size of their population alone.
Acknowledgments
We are grateful to the NHS Information Centre for Health and Social Care for providing data from the National Centre for Health Outcomes Development, NHS Comparators, and the WCC datapacks. The data set and statistical code are available from the corresponding author.
Notes
Funding
Felix Greaves is funded by London Deanery and the National Institute for Health Research (NIHR). Utz J Pape is funded by the NW London NIHR Collaboration for Leadership in Applied Health Research and Care. Christopher Millett is funded by the NW London NIHR Collaboration for Leadership in Applied Health Research and Care and the Higher Education Funding Council for England. The Department of Primary Care and Public Health at Imperial College is grateful for support from the NIHR Biomedical Research Centre scheme, the NIHR Collaboration for Leadership in Applied Health Research and Care scheme; and the Imperial Centre for Patient Safety and Service Quality.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors have declared no competing interests.
Discuss this article
Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss
- Received May 5, 2011.
- Revision received June 14, 2011.
- Accepted July 19, 2011.
- © British Journal of General Practice 2012