DATA AND PERFORMANCE: DIFFERENCES IN APPROACH
At a time when all public sector institutions in England are under increasing pressure to do more with less1 and are often judged on simple numerical scores reflecting aspects of performance, now may be a good time for different sectors to work together to understand the benefits and challenges of collecting and publishing data.
In both health and education, data are collected and published about a multitude of inputs and outcomes, and are frequently used to understand performance, inform policy and practice, as well as to judge and rank institutions and, sometimes, individuals. I have worked in primary and secondary education for 20 years and am now involved in analysing health data. I have been intrigued by some of the differences in the approach to data in these two important public sector areas and this article highlights two areas which may be of particular interest to people working in general practice, focusing on school settings and general practices in England.
UNDERSTANDING THEIR OWN POPULATIONS
Staff in English schools know who they are teaching to an amazing level of detail. They know about pupils’ progress, achievement, and attendance, but also about the interventions they have received and their participation in extra-curricular activities. Pupils are tracked in terms of national and local targets as well as school-based targets. This is known for individual children and year groups. Schools also know this in terms of groups of pupils; for example, whether pupils in receipt of the pupil premium, (extra funding given to schools for children from socioeconomically disadvantaged backgrounds) are making as much progress as pupils who aren’t. In addition, they know about the progress of sub-groups; for example, whether girls in receipt of the pupil premium and who have English as an additional language are making good progress in comparison to their peers. This information is used to support evaluation of current practice, identify areas of concern, and to prioritise new initiatives to improve attainment. Low attainment of particular groups of pupils may lead to small scale, local interventions; such as, using different texts to interest boys in English and bringing in mentors to widen the ambitions of African–Caribbean boys.
While researchers in health commonly take into account demographic information about the practice population, and report on outcomes of different groups of patients, it is not common for practitioners to do this. I would argue that GPs need to be more aware of key demographic data pertinent to health inequalities2 and it has been argued that practices need to increase their knowledge of the populations they serve in order to anticipate what outcomes to expect and to therefore allow a more informed monitoring of general practice.3 Practices are aware of their achievement in terms of national targets, for example the national Quality and Outcomes Framework (QOF), local targets, and the health of individuals, but a deep understanding of their populations and analysis of health attainment by groups of patients is not common. This may be because, despite effective initiatives to improve comprehensive data, important demographic information such as ethnicity, sociodemographic indicators, language, and disabilities are not part of routine patient-level primary care data.4 In contrast, these data are considered key to the efficient working of a school and the databases which underpin school administrative systems are often designed with this type of analysis in mind.5
Having patient-level demographic data readily available in general practice and the IT needed to support this would allow the identification of groups with specific needs to be identified. For example, prescribing, appointment and immunisation patterns could be analysed to determine if particular groups of patients are causing concern and this may lead to a change in treatment approach or how patients are contacted. If particular groups are identified, for example male patients living alone not taking up immunisations, this may influence the way in which these initiatives are promoted within the practice, clinical commissioning group, or local area.
THE INCLUSION OF DEPRIVATION IN PERFORMANCE MEASURES
While the impact of socioeconomic deprivation on individuals is a complex area and causal links are not clear, there is considerable evidence of an association between socioeconomic deprivation and worse outcomes in both health and education and an inter-relationship between the two.6 This has led to initiatives to narrow the gap and reduce health inequalities, which aim to reduce the difference currently experienced by people from differing social backgrounds in educational achievement and health status respectively. This article focuses on the inclusion of deprivation in performance measures, which has proved to be controversial.
Initially, school achievement data, which are used to compile school league tables, did not take into account the background of children, but in 2006 an additional measure known as contextualised value added (CVA) was introduced. This measured the progress of pupils but also took into account ethnicity, sex, poverty, and special educational needs. In 2010 the Department for Education described the idea as morally wrong and the measure was abandoned.7
The majority of health services research includes a measure of deprivation, either as a confounding factor or as an area of focus, and some health indicators take deprivation into account. For example, while both the Hospital Standardised Mortality Ratio (HSMR) and the Summary Hospital Mortality Index (SHMI) compare observed numbers of deaths in hospitals to those expected, the HSMR adjusts for deprivation but this is not the case for the SHMI.8 However, general practice performance data,9 such as QOF and patient experience scores, are not adjusted for deprivation or any of the other factors, such as disability, which may have an impact on general practice performance.
The way forward is not simple, there is concern that by not adjusting measures for deprivation, and other factors, practices and schools may justify lower performance by an explanation that their practice or school is different and avoid enrolling patients and children who are deemed hard to treat or teach.10 In justifying the abolition of CVA measures, concern was expressed that schools should not expect different progress from different groups of pupils on the basis of their ethnic background or family circumstances. It is clearly important not to encourage complacency or to create lower standards in general practices and schools serving disadvantaged areas. But arguably, it is also important to understand that deprivation, as well as other contextual factors, is an additional barrier faced by some schools and general practices. Fiscella and colleagues,11 in their discussion on whether to adjust by socioeconomic factors, recommend that performance rates should be reported by groups, poor versus non-poor, English speaking versus not. They propose that such stratified reporting provides the most direct view of healthcare disparities and also supports the planning of targeted initiatives designed to mitigate these disparities. In education, achievement is now published for disadvantaged pupils and other pupils, which emphasises the gap in achievement between these two groups. This returns us to the need to understand the populations we work with.
It seems likely that neither schools nor general practices will have a simple solution to this challenge, but perhaps with a shared understanding of the aims and uses of performance measures health and education professionals can work together to generate a useful way forward.
DIFFERENCES
Sharing approaches to using data in education and health implies a clear understanding of the differences between schools and general practices. Most schools are smaller than general practices in terms of the number of patients/pupils on their lists, but not in the number of staff. Even the biggest secondary schools only start to approach the size of the smallest general practices. However, a secondary school of 1800 pupils could employ over 250 adults, in contrast to less than 10 staff members for a similar sized practice. In addition, there is an expectation that schools will see their pupils at least 180 times in a year, which is clearly a very different picture to any healthcare setting. General practices only really work with individuals, and therefore health care is individualised.
In contrast, schools and their curricula are organised on a group basis and analysing data in this way may therefore seem more intuitive and, while schools are developing more individualised approaches, group-based analysis has proved an effective tool. There are, of course, many more differences, which would need to be explored for effective collaborative working.
CONCLUSION
This article is aimed at people working in health care and has focused on two areas in which education and health could work together to ensure data are used more effectively; there are of course many other areas.
A similar article aimed at teachers and other education professionals could highlight the more varied ways in which data are presented in health and the apparently better understanding of the impact of common cause variation, particularly on small samples.12 A discussion of the role of randomised control trials to contribute to evidenced-based practice and the professional as researcher may also be important areas for discussion.
One of the main challenges in both health and education, at a time when data are used to judge and rank institutions, is to ensure that data are, instead, used to inform service review and improvement through a considered and collaborative approach,13 without increasing the pressure on already over-stretched services.
Acknowledgments
The author would like to thank Professors David Jones and Richard Baker of the University of Leicester for their constructive comments on early drafts which greatly improved the manuscript as well as the comments of reviewers which have shaped the final version.
Notes
Provenance
Freely submitted; externally peer reviewed.
Funding
This article was written while Kate Honeyford was working at the University of Leicester funded by Leicester Clinical Commissioning Group. The views expressed are those of the author and not necessarily those of the University of Leicester or Leicester CCG.
Competing interests
The author has declared no competing interests.
- © British Journal of General Practice 2016