Identifying primary care quality indicators for people with serious mental illness: a systematic review

Background Serious mental illness (SMI) — which comprises long-term conditions such as schizophrenia, bipolar disorder, and other psychoses — has enormous costs for patients and society. In many countries, people with SMI are treated solely in primary care, and have particular needs for physical care. Aim The objective of this study was to systematically review the literature to create a list of quality indicators relevant to patients with SMI that could be captured using routine data, and which could be used to monitor or incentivise better-quality primary care. Design and setting A systematic literature review, combined with a search of quality indicator databases and guidelines. Method The authors assessed whether indicators could be measured from routine data and the quality of the evidence. Results Out of 1847 papers and quality indicator databases identified, 27 were included, from which 59 quality indicators were identified, covering six domains. Of the 59 indicators, 52 could be assessed using routine data. The evidence base underpinning these indicators was relatively weak, and was primarily based on expert opinion rather than trial evidence. Conclusion With appropriate adaptation for different contexts, and in line with the relative responsibilities of primary and secondary care, use of the quality indicators has the potential to improve care and to improve the physical and mental health of people with SMI. However, before the indicators can be used to monitor or incentivise primary care quality, more robust links need to be established, with improved patient outcomes.


INTRODUCTION
Serious mental illness (SMI) includes schizophrenia, bipolar disorder, and other psychoses (defined by International Classification of Diseases [ICD-10] 1 categories F20-F31, and including schizophrenia spectrum and other psychotic disorders together with bipolar and related disorders in DSM-5). 2 SMI is linked with poor health outcomes, high healthcare costs, and high disease burden. 3,4 People with SMI have, on average, a 20-year lower life expectancy, mostly due to preventable causes. [5][6][7][8] The global morbidity study attributed 3.5% of total years lost to disability to schizophrenia and bipolar disorder combined. 9 SMI is also associated with increased treatment costs 10 and hospitalisations. Yet, around a third of people with SMI in the UK are treated solely in primary care, 11 and are in longterm contact with primary care services more often than the general population. 12,13 Even in countries with very well developed secondary mental health care systems, primary care can make a key contribution to the care of people with SMI. 14 The quality of primary care for people with mental health problems is therefore of international concern. 15, 16 In the UK, a national pay-for-performance scheme, the Quality and Outcomes Framework (QOF), exists to financially reward family practices for achieving quality targets for patients with long-term conditions. The SMI quality indicators in the QOF cover both mental health specific care (for example, monitoring lithium levels) and more general physical care (such as routine health checks). QOF indicators are for high-priority disease areas for which primary care has principal responsibility for ongoing care, and where there is good evidence that improved primary care will have health benefits. However, the QOF may neglect important unmeasured aspects of quality of care, 17 and the incentives may result in tunnel vision, 18 or a focus on activities that are prioritised at the expense of other non-incentivised activities. 19,20 For example, the QOF focuses more on physical than mental health, because this is generally easier to measure.
The authors performed a systematic review of the literature and interrogated international databases to identify potential quality indicators that could supplement or replace indicators already included in the QOF for people with SMI, and which could potentially be incentivised in primary care. The authors included indicators that appeared in earlier versions of the QOF but were subsequently dropped from the scheme when it was reduced in scope to reduce workload. These indicators were included on the grounds that they remain valid measures of quality of care, and Research Abstract Background Serious mental illness (SMI) -which comprises long-term conditions such as schizophrenia, bipolar disorder, and other psychoses -has enormous costs for patients and society. In many countries, people with SMI are treated solely in primary care, and have particular needs for physical care.

Aim
The objective of this study was to systematically review the literature to create a list of quality indicators relevant to patients with SMI that could be captured using routine data, and which could be used to monitor or incentivise better-quality primary care.

Design and setting
A systematic literature review, combined with a search of quality indicator databases and guidelines.

Method
The authors assessed whether indicators could be measured from routine data and the quality of the evidence.

Results
Out of 1847 papers and quality indicator databases identified, 27 were included, from which 59 quality indicators were identified, covering six domains. Of the 59 indicators, 52 could be assessed using routine data. The evidence base underpinning these indicators was relatively weak, and was primarily based on expert opinion rather than trial evidence.

Conclusion
With appropriate adaptation for different contexts, and in line with the relative responsibilities of primary and secondary care, use of the quality indicators has the potential to improve care and to improve the physical and mental health of people with SMI. However, before the indicators can be used to monitor or incentivise primary care quality, more robust links need to be established, with improved patient outcomes. continue to be included in the broader National Institute for Health and Care Excellence (NICE) indicators menu. A major focus of the analysis was the source of the data on which the indicators were based. Those requiring primary data collection -for example, via surveys of patients or health professionals, or retrospective auditing of patient records -would be very challenging to incorporate into incentive schemes such as the QOF, whereas those based on routinely available data would, in principle, be more feasible to establish.
Previous literature reviews on quality indicators have focused on SMI in secondary care, 22,23 whereas this study (to the authors' knowledge) is the first to focus specifically on people with SMI in primary care. Identifying indicators of primary care quality for people with SMI could help to strengthen the evidence base and shed light on neglected areas of care, as well as providing the basis for incentive schemes aimed at improving quality.

METHOD
A systematic review of primary care quality indicators for people with SMI was conducted with the aim of identifying quality indicators in addition to those already included in the QOF, either in the past or currently.

Inclusion and exclusion criteria
The authors searched for published examples of potential quality indicators that could readily be collected in primary care with reference to routine data. Search terms were identified by an information specialist in conjunction with the project team. Included papers had the terms serious mental illness AND primary care AND quality indicator, including alternative spellings and synonyms. Studies on children or covering non-psychotic illnesses, for example, severe depression or anxiety disorders, were excluded. All studies from January 1990 to February 2015 were considered for inclusion. No language restrictions were applied, although all search terms were in English, and all studies in English, German, Dutch, and Afrikaans were considered due to authors' language knowledge. The base search was constructed using MEDLINE and adapted to the other resources. The following databases were searched: Applied Social Sciences Index and Abstracts (ASSIA); CENTRAL; Cochrane Database of Systematic Reviews; Conference Proceedings Citation Index-Science (CPCI-S); Database of Abstracts of Reviews of Effects (DARE); EMBASE; Ovid MEDLINE ® In-Process & Other Non-Indexed Citations and Ovid MEDLINE ® ; PsycINFO; and MEDLINE. The full strategy for MEDLINE as a template is available in Appendix 1.
Additionally, previous reviews with overlapping aims were searched, and authors were contacted to ask for their indicators (most notably Stegbauer et al 22 and Großimlinghaus et al 24 ). The quality indicator database of the Agency for Healthcare Research and Quality (AHRQ) 25 was also searched for indicators relevant to primary care. The final selection of indicators was informed by the views of the study steering committee, which included service users.

How this fits in
This is the first systematic review of indicators of primary care quality for patients with serious mental illness (SMI). The study identifies 59 quality indicators in six domains, the majority of which could be monitored using routine primary care data. A key domain is the focus on physical health care. Consideration of the use of a broad set of quality of care indicators may support the improvement of the mental and physical health of this patient group.  System contact: number of patients in contact with the treatment system Routine data 23 15 Surveillance to prevent relapse Routine data 27 16 Crisis management and out-of-hours services Routine data 28 17 Access to services and range of services Routine data 27 18 Family care -record of families living with person with schizophrenia Primary data 26 19 Duration of untreated psychosis: number of recently diagnosed patients Routine data 23,34 20 Waiting time between registration and start of treatment Routine data 23 Medicines management

Study selection
Titles of papers were first reviewed by five authors, and all studies that had been identified as potentially relevant by two members of the research team were included in the abstract screening process. All abstracts were screened by six authors, and full papers were obtained if two members of the team judged the abstract potentially relevant or in scope, that is, covering serious mental illness, primary care, and quality indicators. Full papers were divided into four groups and independently reviewed by four pairs of authors. The focus of the selection was to identify papers that included relevant quality indicators that could be applied in primary care. It was evident that the definition of primary care varies between different countries, so the authors included indicators with elements of shared care between primary and specialist settings (for example, prescribing and monitoring of antipsychotic medication), while acknowledging that, in some countries, those indicators may be more applicable to secondary care. The search strategy complied with the PRISMA checklist (Appendix 2).

Data extraction and analysis
From each paper, a short description of each indicator was extracted, and the descriptions for similar indicators were merged. After reviewing the general areas covered by the indicators, they were grouped into six Table 1 continued. Quality of care indicators identified for people with serious mental illness 33 Comprehensive assessment of comorbid psychiatric conditions and response to treatment Routine data ü 25 34 Reassess severity of symptoms Routine data ü 44 35 Examined for duration of untreated psychosis Primary data 34 36 Delayed diagnosis Primary data 45 37 Informal carer contacts Primary data 27 38 Information on employment status Primary data 26 Physical health assessment and care  combined with a short list of potential quality measures Holden 26 This study audited 16 GPs on their care for 266 patients with schizophrenia and observed that the 3 audit led to improved recording of a range of quality indicators Swartz, MacGregor 31 The authors of this paper argue that in South Africa the role of mental health nurses has 3 been altered to focus on violence, substance misuse, and HIV/AIDS, and should be refocused on psychiatry care in the primary care setting Ruud 34 The author summarises the literature on quality of care in mental health services in 3 Norway in 2008-2009 Highet, McNair, Interviews with 49 patients with bipolar to describe experience in primary care in Australia. 3 Thompson, et al 45 Eight themes for improvement of the primary care experience are outlined Lader 35 Expert review of the standards of care in schizophrenia to reduce side effects while 3 achieving best treatment outcomes Haro, Salvador-Carulla 36 Observational study following 11 000 patients who were on or changing antipsychotic 2 medication to determine the best course of treatment with respect to symptoms, quality of life, social functioning, and other outcomes Caughey, Kalish Ellett, Wong 38 Development, expert review, and assessment of the evidence base for, and validity of, 3 medication-related indicators of potentially preventable hospitalisations Some of the indicators may overlap the domain description boundaries as they are not intended to be rigid boundaries. Given the main focus of the study, the authors decided whether each indicator could, in principle, be measured from routine data or whether primary data collection would be necessary. Furthermore, the authors checked whether the identified indicators had ever been included in the QOF. They also assessed the quality of the evidence of the included studies using an adaptation of the GRADE guidelines, 52 and rated the quality of the evidence as high (systematic reviews or randomised control trials), moderate (nonrandomised control studies or unsystematic reviews), low (expert opinion or uncontrolled studies), or not applicable (measure was extracted from the grey literature).

RESULTS
In total, 1847 studies and further database sources were identified using the search. The split was ASSIA (34), CENTRAL (96), Cochrane (12), Conferences Proceedings (125), DARE (28), EMBASE (738), Ovid MEDLINE (537), PsycINFO (271), and six further database sources (AHRQ, 25 Stegbauer et al, 22 Großimlinghaus et al, 24 Parameswaran et al, 23 and NICE). 30,37 After removing duplicates using bibliographic software (EndNote and Zotero), 1303 records remained. Title screening reduced this to 356, excluding those that were not about quality indicators, or primary care, or mental illness, or were not included in the definition of SMI (for example, depression or substance misuse disorders). Abstract screening reduced the records to 113, with similar reasons for exclusion. Finally, from those 113 records, 86 were excluded, and 27 records were included in the review (Figure 1, PRISMA flow diagram). From these 27 records, a final set of 59 different indicators was extracted. Table 1 shows the final list of indicators. Of the 59 indicators, 52 could potentially be assessed using routine data, and seven would require primary data collection from patients or professionals. Of these 59 indicators, 17 are, or have previously been, included in the QOF. A large proportion of the indicators relevant to primary care are in the physical health domain. Another large subset of indicators relate to the process of receiving care, for example, continuity of care, access to services, and frequency of contacts. Table 2 shows the quality of evidence of the included studies from which the indicators were drawn. Two studies were rated as high quality (Cochrane or systematic review, randomised control trial); three as moderate (non-randomised study or unsystematic review); 19 as low quality (expert opinion, uncontrolled studies); and three were of uncertain quality, having been identified from the 'grey' literature (for example, (non-)government organisations' documents or databases).
Only a very few randomised control trials (RCTs) have evaluated quality indicators. Two RCTs were reviewed in Cimo et al, 50 producing evidence on the effectiveness of lifestyle interventions for people with type 2 diabetes and schizophrenia or schizoaffective disorder. However, more often, indicators were based on expert consensus or small cross-sectional studies.
Many of the indicators identified were derived from a database of indicators produced by the US Agency for Healthcare Research and Quality (AHRQ), 25 and the strength of evidence underpinning the individual indicators is variable.

DISCUSSION Summary
To the authors' knowledge, this is the first attempt to identify in a systematic way potential indicators of quality of primary care for people with SMI. Although the authors identify over 50 indicators that could potentially be captured and monitored using routine data, crucially, they note that the quality of the available evidence underpinning the indicators is relatively weak.

Strengths and limitations
The feasibility of collecting data for any set of quality indicators will vary across different healthcare systems. Many countries have insurance or other systems, which routinely collect activity data in primary care. Some indicators are likely to require more effort to collect (for example, patient questionnaires for perceived continuity of care), and in many cases even routine data collection can prove very challenging. This study focused specifically on finding indicators that could be monitored at relatively low cost to the healthcare system. The list of quality indicators identified in this study is much broader and more encompassing than the current list of indicators contained in the QOF SMI domain. However, some of the criticisms inherent in the use of quality metrics would remain even if indicators from this broader list were adopted. These include: measuring only what can be measured (in routine data) at the expense of other measures that matter, for example, 'softer' measures such as the quality of relationships or the quality of communication; 53 the risk of prioritising some activities at the expense of other non-incentivised activities; 54,55 and the wider impacts of financial incentives and excessive measurement on provider motivation and behaviour. 56 Moreover, there are gaps in the literature and in the indicators identified, meaning that the service user perspective is not well represented. There is also an absence of quality indicators around aspects of the social environment, such as the stability of housing for people with SMI. Although such factors are important, and may well influence health outcomes, the extent to which primary care could influence these factors may be very limited and hence it may not be appropriate to hold primary care practitioners responsible for improving quality in these domains. The authors also acknowledge that there is an extensive literature in related areas of research that will also refer to very similar quality indicators, 57 but the search terms were designed to focus on the specific area of interest, and screened out studies where the focus was broader. Finally, the search excluded non-published indicators and those written in languages other than those listed earlier.

Implications for research and practice
In the UK, to be included in the QOF, quality indicators must be supported by NICE evidence-based clinical guideline recommendations or evidence from systematic reviews. This, along with the need to maintain a manageable panel of indicators, explains why the large majority of indicators identified are not currently part of the QOF. The downside of the QOF approach is that recommendations based on expert consensus are not put forward for inclusion, despite the fact that a body of informed experts would support a prima facie rationale for including them. In contrast, the combined views of experts and patients underpin bestpractice guidance for those commissioning mental health services in the UK, covering many of the domains identified in this review, suggesting scope for a similar approach to be taken with respect to the QOF. 58 The adoption of indicators based on expert and patient consensus must ultimately be supported by evidence on cost-effectiveness, but this also applies to indicators based on higher levels of evidence. 59 Donabedian's 60 conceptual framework of quality of care suggests indicators can usually be divided into three subcategories: structure, process, and outcome measures. To date, the evidence for apparent process improvements under incentive schemes leading to improved patient outcomes is mixed. The vast majority of indicators included in this review relate to processes of care, and, although aspects of process are highly relevant, especially to patients, it is important to establish whether quality indicators also promote improved health outcomes. If so, there is a case for their inclusion in the QOF and other initiatives aiming to improve the care of people with SMI. For physical conditions, improvements in processes of care in primary care settings have been found to be associated with modest improvements in intermediate outcomes (for example, cholesterol levels) 61 and quality of life, 62 but associations with patient outcomes such as emergency hospital admission are weaker. 63 For serious mental illness, the evidence is much more limited and suggests that higher provider performance on processes may not be associated with better patient outcomes. 17 Many of the indicators identified in the study relate to aspects of physical care. People with SMI are at higher risk of physical ill health (particularly diabetes, and cardiovascular and respiratory disease), so clearly focusing on these aspects could help reduce the associated excess morbidity and mortality. 64 People with SMI are vulnerable, with significant needs for care that may be missed or undertreated, leading to years spent with disabling morbidity and premature mortality. Viron et al 14 emphasised that, in the US, as elsewhere: 'As frontline clinicians, primary care providers have the potential to reduce the health disparities experienced by this population.' Consideration of the use of a broader set of quality indicators, including those focusing on physical care, may therefore be a positive step. Given the increased risk of diabetes, cardiovascular disease, and respiratory disease in this population, ongoing primary care for people with SMI should focus on disease prevention through tackling obesity and smoking. Similarly, a large set of indicators relate to processes of care, including ongoing contact with relevant services. Targeting comprehensive primary care to people with SMI can also play a crucial role in promoting their engagement with appropriate specialised mental and physical healthcare services, helping them to reach their full potential.

Funding
The National Institute for Health Services & Delivery Research (HS&DR) programme funded this research (project no. 13/54/40). The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HS&DR programme, the National Institute for Health Research, the NHS, or the Department of Health.

Ethical approval
Not applicable.

Provenance
Freely submitted; externally peer reviewed.