Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Effectiveness of Case Management for 'At Risk' Patients in Primary Care: A Systematic Review and Meta-Analysis

  • Jonathan Stokes ,

    jonathan.stokes-3@postgrad.manchester.ac.uk

    Affiliation NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre, Manchester Academic Health Science Centre, University of Manchester, Manchester, United Kingdom

  • Maria Panagioti,

    Affiliation NIHR School for Primary Care Research, Centre for Primary Care, Manchester Academic Health Science Centre, University of Manchester, Manchester, United Kingdom

  • Rahul Alam,

    Affiliation NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre, Manchester Academic Health Science Centre, University of Manchester, Manchester, United Kingdom

  • Kath Checkland,

    Affiliation NIHR School for Primary Care Research, Centre for Primary Care, Manchester Academic Health Science Centre, University of Manchester, Manchester, United Kingdom

  • Sudeh Cheraghi-Sohi,

    Affiliation NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre, Manchester Academic Health Science Centre, University of Manchester, Manchester, United Kingdom

  • Peter Bower

    Affiliation NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre, Manchester Academic Health Science Centre, University of Manchester, Manchester, United Kingdom

Abstract

Background

An ageing population with multimorbidity is putting pressure on health systems. A popular method of managing this pressure is identification of patients in primary care ‘at-risk’ of hospitalisation, and delivering case management to improve outcomes and avoid admissions. However, the effectiveness of this model has not been subjected to rigorous quantitative synthesis.

Methods and Findings

We carried out a systematic review and meta-analysis of the effectiveness of case management for ‘at-risk’ patients in primary care. Six bibliographic databases were searched using terms for ‘case management’, ‘primary care’, and a methodology filter (Cochrane EPOC group). Effectiveness compared to usual care was measured across a number of relevant outcomes: Healthself-assessed health status, mortality; Costtotal cost of care, healthcare utilisation (primary and non-specialist care and secondary care separately), and; Satisfactionpatient satisfaction. We conducted secondary subgroup analyses to assess whether effectiveness was moderated by the particular model of case management, context, and study design. A total of 15,327 titles and abstracts were screened, 36 unique studies were included. Meta-analyses showed no significant differences in total cost, mortality, utilisation of primary or secondary care. A very small significant effect favouring case management was found for self-reported health status in the short-term (0.07, 95% CI 0.00 to 0.14). A small significant effect favouring case management was found for patient satisfaction in the short- (0.26, 0.16 to 0.36) and long-term (0.35, 0.04 to 0.66). Secondary subgroup analyses suggested the effectiveness of case management may be increased when delivered by a multidisciplinary team, when a social worker was involved, and when delivered in a setting rated as low in initial ‘strength’ of primary care.

Conclusions

This was the first meta-analytic review which examined the effects of case management on a wide range of outcomes and considered also the effects of key moderators. Current results do not support case management as an effective model, especially concerning reduction of secondary care use or total costs. We consider reasons for lack of effect and highlight key research questions for the future.

Review Protocol

The review protocol is available as part of the PROSPERO database (registration number: CRD42014010824).

Introduction

Many health care systems currently face significant pressures resulting from both increasing numbers of older patients with multiple long-term conditions (multimorbidity), and pressure to reduce health care budgets or provide more efficient use of current resources [1].

To relieve these pressures, many policy makers and health system planners advocate ‘integrated care’ [1, 2].

Integrated care is a complex concept. Broadly, it is designed to “create connectivity, alignment and collaboration” [3]. A number of different methods can be used to achieve these inter-connections, and they can occur at multiple ‘levels’ of the health system (e.g. financing, resource management, service delivery—see Fig 1). Outcomes of effective integration of care are presumed to be better patient experience and outcomes, as well as greater efficiency [4] (i.e. patient satisfaction; health; and cost-effectiveness), therefore potentially addressing two of the major system pressures simultaneously.

thumbnail
Fig 1. Examples of popular methods to ‘integrate’ care [3] within the health system [5].

https://doi.org/10.1371/journal.pone.0132340.g001

A popular model of ‘integrated care’ at the service delivery level is ‘case management’ in primary care [6, 7]. Case management has been defined as:

“a collaborative process of assessment, planning, facilitation, care coordination, evaluation, and advocacy for options and services to meet an individual’s and family’s comprehensive health needs through communication and available resources to promote quality, cost-effective outcomes” [8].

Variations exist in the delivery of case management. However, there are common components [6]:

  • case-finding (identifying those ‘at risk’ who require case management, usually through prediction of high costs in the future [9, 10])
  • assessment of the needs of the individual patient, and care planning (individualised care plan bringing together details of patient’s personal circumstances with health and social care needs, and aiming to match these needs with service provision)
  • care co-ordination (navigational role of case manager involving continual communication with patients, carers, professionals and services e.g. medication management, self-care support, care advocacy and negotiation; with regular review, monitoring and adaptation of the care plan)

Case finding of ‘at-risk’ individuals can be done in three ways [11]:

  1. clinical judgement (expert opinion),
  2. threshold modelling (defining a set of rules e.g. number of previous hospital admissions, which alert the practitioner that the patient is at risk), or
  3. using a predictive risk tool (where an algorithm is used to attempt to predict those patients who are at risk of a defined event)

In theory, the case management process may increase efficiency by reducing unnecessary contacts with the health system, including fragmented routine contacts, as well as emergency contacts caused by potentially preventable exacerbations. The goal is to better co-ordinate care, offering individually-tailored contacts and care planning.

Primary care is a suitable context for integrated care due to its place at the heart of the health system [12]. It is argued that increasing care in the community setting will facilitate cost savings compared to expensive hospital overheads [13]. In many health systems, primary care acts as a ‘gatekeeper’ to the rest of the system [14] and primary care practitioners should be particularly suited to managing and co-ordinating care for multiple health problems, compared to specialist physicians [15].

The potential benefits of case management have led to adoption in practice in many countries [6]. For example, in the United Kingdom, recent changes to the NHS GP contract (under the Unplanned Admissions Enhanced Service section), require a minimum of 2% of the risk-identified population to be proactively case managed [16]. In the USA, a number of health insurers and health maintenance organisations offer case management to patients with long-term conditions, for example the ‘Guided care’ and similar programmes [6].

It is important that the provision of case management in primary care should be based on rigorous evidence. While many descriptive reviews exist examining specific types of case management in primary care (such as nurse-led case management [17]), there is no published systematic review of a range of current case management models for high risk individuals in primary care that provides a formal meta-analytic review of its effectiveness across a range of relevant outcomes.

Objectives

  1. To synthesise the evidence for the effectiveness of case management in primary care for ‘at risk’ patients
  2. To explore whether the effectiveness of case management in primary care is moderated by the particular model of case management implemented, context, and study design.

Methods

The methods and results for this review are reported in line with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. The review protocol is available as part of the PROSPERO database (registration number: CRD42014010824).

Eligibility criteria

Studies were included in this review if they met the following criteria:

  • Population: Adults (18+) with long-term condition(s)
    (While prevalence of multimorbidity (i.e. ‘complex’ cases) is highest in the elderly, the absolute numbers affected are greater in those below 65 [18])
  • Intervention:
    • Adopting methods to identify ‘at-risk’ patients to receive the case management, with the aim of preventing acute exacerbations of symptoms, and/or secondary care utilisation among those at higher risk
    • Case management, including all of the following activities: case-finding; assessment; care planning; care co-ordination; regular review, monitoring and adaptation of the care plan
    • Primary care/community-based management (regardless of where the case was first identified)
  • Comparison: usual care or no-case management
  • Outcome categories: Healthself-assessed health status, mortality; Costtotal cost of care, healthcare utilisation (primary and non-specialist care and secondary care separately), and; Satisfactionpatient satisfaction
  • Study design: Quantitative empirical research, meeting Cochrane Effective Practice and Organisation of Care (EPOC) Group study design criteria: randomised controlled trials (RCTs), non-randomised controlled trials (nRCTs), controlled before and after studies (CBA), and interrupted time series (ITS)

Exclusion criteria:

  • Case management targeted solely at care for patients with mental health problems, although mental health conditions could be included where they were co-morbidities alongside physical long-term conditions
  • Hospital discharge planning (short-term management to facilitate the transition from hospital to home [19])
  • Non-English language papers and grey literature

Search Strategy

Six main electronic bibliographic databases were searched for potential studies from inception until end of April 2014: MEDLINE (Ovid), EMBASE (Ovid), CINAHL, Cochrane Register of Controlled Trials (CENTRAL), Health Management Information Consortium (Ovid), and CAB Global Health (Ovid),. The search strategy used three key blocks of terms (including subject headings as well as text-words): 1) Case management2) EPOC methodology filter[20]3) Primary care filter[21]. S1 Appendix shows an example of the full search strategy for the MEDLINE database.

Hand searches of the reference lists of included papers, plus previous relevant systematic reviews [17, 2230] supplemented the database searches.

Results from the above searches were combined in an Endnote library, and duplicates were removed (n = 2186) prior to study selection.

Study Selection

Study selection was carried out in two stages. First, titles and abstracts of the identified studies were screened in full by the first author. A proportion of these titles and abstracts (10%) were then independently screened by a second author (kappa coefficient = 0.78). Following this initial screening, the full texts of the identified articles were retrieved, and reviewed against the inclusion/exclusion criteria. Forty percent (n = 106/266) of the full text screening was carried out by two reviewers independently. Inter-rater reliability was high (kappa coefficient = 0.81), and any disagreements (n = 7) were resolved by group discussion (resulting in 4 included, and 3 excluded). The remaining full text screening was completed by the first author alone.

Data extraction

A data extraction form was formulated using Microsoft Excel. The form was initially piloted on two randomly selected studies. The following descriptive data were extracted for included studies:

  • Patient: target population; total sample size (intervention/control); proportion of males; average age; average baseline number of long-term conditions; average baseline number of emergency department visits/ hospital admissions in previous year
  • Intervention: name of the case management model; brief description of model; intensity of intervention; multidisciplinary team(and specific members) or single case manager; primary case manager; primary location of case management; risk stratification model used; whether there was 24-hour availability of a case manager; caseload; whether the case manager received training in the intervention protocol; reimbursement method
  • Context: country was used to define the ‘strength’ of primary health care, classified according to Starfield & Shi’s work [31]
  • Outcome categories: Healthself-assessed health status, mortality; Costtotal cost of care, healthcare utilisation (primary and non-specialist care and secondary care separately), and; Satisfactionpatient satisfaction
  • Study design: design; study duration; unit of analysis; eligibility criteria; type of control group

On a separate sheet, relevant quantitative data for the meta-analysis were extracted (see quantitative analysis section below). Where adjusted and unadjusted results were both presented, the result adjusting for the most potentially confounding variables was extracted.

25 percent (n = 9 studies) of the data were extracted by two researchers working independently. The agreement was high (kappa coefficient = 0.85, across 326 data points), and the remainder of the data were extracted by the first author, and the accuracy of extraction verified by a second reviewer.

Quality Assessment

In the original protocol, we predicted having to use multiple measures of risk of bias to suit the various study types included in the eligibility criteria. However, having identified the full text articles and study designs represented, it became clear that it would be possible and preferable to use a single quality assessment tool, the EPOC risk of bias tool [32], better allowing comparison of quality across the included studies. The EPOC risk of bias tool encompasses nine standardised criteria to judge the quality of all RCTs, nRCTs, CBA and ITS studies. Each of the nine criteria is judged on a 3-point scale, corresponding to: low risk, unclear risk, and high risk. To ease comparison between studies, the total number of criteria met by each included study was also reported. Those studies at high risk of bias (fulfilling three or less criteria) were removed from the synthesis for sensitivity analysis.

Quantitative Analysis

Meta-analysis was carried out on six outcome categories related to the three main health system goals [5]. These were: Healthself-assessed health status, mortality; Costtotal cost of care, healthcare utilisation (primary and non-specialist care and secondary care separately), and; Satisfactionpatient satisfaction. Table 1 clarifies which measures were included within each of these outcome categories.

In addition to the outcomes specified in the original protocol, we also attempted to extract data related to the outcome category of ‘patient safety’: admissions for ambulatory care sensitive conditions [33], and polypharmacy (simple count of medications). However, none of the included studies reported these outcome measures, so results could not be synthesised.

Meta-analysis was carried out on each outcome, distinguishing between effects over the short-term (0–12 months), and longer-term (13+ months). Meta-analysis used the standardised mean difference measure, based on the mean of the case management group minus mean of the control group, divided by the pooled standard deviation [34]. When multiple measures were available for a single study within a certain outcome category, the median effect was used, as recommended in the literature [35] (e.g. for the outcome of self-assessed health status, if a measure of activities of daily living, of restricted activity days, and a measure of QALYs were all available for a given study, the effect size for each of these would be calculated, and the median standardised mean difference would represent this studies’ overall effect for this outcome). We adopted Cohen’s rule of thumb for interpreting effect sizes, i.e. that 0.2 indicates a small effect, 0.5 a medium, and 0.8 a large effect [36].

Heterogeneity in the outcomes was assessed using the I2 statistic, interpreted as the percentage of total variation in the study estimates due to heterogeneity [37]. A random effects model was chosen to present the pooled effect results based on the relatively high level of heterogeneity assumed between studies evaluating a complex intervention in a variety of service contexts.

Funnel plots were performed to assess small sample bias (which may bean indicator of publication bias), but only for those outcomes drawing on 10 or more studies, as recommended [38]. Egger’s test of small-study effects was additionally performed to quantify observations in the funnel plots [39].

As a complex intervention, context may be of some importance when assessing case management [40]. Subgroup analyses were performed where 10 or more studies contributed effect size data. The pre-specified variables were:

  • Context: strength of primary health care orientation of the health system (low versus intermediate/ high)
  • Type of case management: multidisciplinary team (MDT) versus single case manager; type of risk tool used (judgement versus threshold/ predictive risk modelling); inclusion of a social worker in the case management (versus absence)
  • Study design: RCT versus non-RCT

Table 2 discusses the justifications for these choices of subgroup.

Statistical significance between subgroups was judged by overlap of each subgroup’s pooled effect (i.e. overlap of confidence intervals between subgroup effects indicates no significant difference) [47].

The majority of effect sizes were calculated using the Metaeasy software add-in for Microsoft Excel (version 1.0.4) [35]. The Metaeasy software allows standardisation of effect size from a variety of input parameters (dichotomous, continuous or both data types), according to eight possible methods described by the Cochrane Collaboration [48]. When multiple methods are available for a single outcome, methods are prioritised according to expected statistical precision [35]. To maximise included results, the Metaeasy effect size calculation methods were supplemented by methods developed by Lipsey & Wilson [49], with a calculator available at http://www.campbellcollaboration.org/resources/effect_size_input.php. Effect directions were transformed so that a positive effect represented favouring case management for all outcome measures. These final effect sizes and their standard errors were then input to STATA together with relevant study information for the subgroup analysis. The final meta-analyses were then run on STATA (version 13) [50] using the metan command [51]. Funnel plots were prepared using the metafunnel command [52], and the Egger test with the metabias command [53].

Sensitivity analyses and Multiple comparisons

Two separate post-hoc sensitivity analyses were conducted in addition to the specified PROSPERO protocol. Studies were removed from analysis if they were:

  1. at high risk of bias (meeting 3 or less of the criteria for assessment of study quality)
  2. set in a Veteran’s health setting, where over 90% of the patients were males.

With multiple comparisons, the chances of inflating type I errors is increased [54]. We therefore used the Holm-Bonferroni adjustment [55] for multiple comparisons to identify potential false positive results.

Results

Fig 2 shows the PRISMA flow diagram, with the studies included/excluded at each stage of the screening process. 36 unique studies were finally included in the meta-analyses.

Characteristics of included studies

Unsurprisingly, since the majority aimed at an elderly population, the average age in nearly all studies was high (mean age: 75.7, range of mean age: 49.0 to 87.3).

Table 3 shows the demographic characteristics of the studies. Of note, 5 (14%) provided case management to a population composed of over 90% males, carried out in veterans’ settings. Inadequate information was provided across studies on baseline number of long-term conditions, and baseline utilisation of emergency and specialist services.

thumbnail
Table 3. Demographics of included studies.

N/R = Not Reported; N/A = Not Applicable.

https://doi.org/10.1371/journal.pone.0132340.t003

Table 4 summarises the potentially relevant contextual factors. Of the 36 studies included, the majority were from the USA (n = 21, 58%). When classified according to relative strength of primary care orientation, 23 studies (64%) were set in a system with low strength of primary care, and 13 (36%) in an intermediate, or high strength system. Three studies (8%) were targeted at patients with specific conditions (COPD/chronic heart failure) while the majority targeted populations more broadly on frailty, chronic illness or high utilisation (92%).

A brief qualitative description of each intervention is also provided in Table 4. Table 5 compares some of the key attributes of each intervention more directly. Many criteria highlighted as key to understanding integrated care interventions [9] were inadequately reported which limited their utility for analysis. However, the type of risk tool, whether the case management was carried out by a MDT or single case manager, and the inclusion of a social worker in the case management could be recorded for all studies. The majority of studies used a ‘threshold’/’predictive risk modelling’ risk assessment tool (n = 32, 89%), with only 4 (11%) using clinical judgement. Twenty-one studies (58%) employed MDT case management. A social worker was involved in the case management in 12 studies (33%).

Methodological Quality

The majority of studies (n = 28, 78%) used an RCT. The length of follow-up in the studies varied, with a range of 6 to 60 months. Table 6 shows the methodological quality according to the nine criteria of the EPOC risk of bias tool. The studies were of variable quality, with 64% (n = 23) fulfilling seven or more criteria, 30% (n = 11) fulfilling between four and six criteria, and 6% (n = 2) fulfilling three or less.

Primary analyses

Figs 38 show the results of the primary meta-analyses for the six outcome categories assessed (both short- and long-term).

thumbnail
Fig 3. Forrest plot for self-assessed health status outcome.

Effect estimates are the standardised mean difference, where the solid vertical line at 0 indicates no effect. Effect estimates are based on a random-effects model. Each subtotal shows the overall effect estimate for the time-period indicated.

https://doi.org/10.1371/journal.pone.0132340.g003

thumbnail
Fig 4. Forrest plot for mortality outcome.

Effect estimates are the standardised mean difference, where the solid vertical line at 0 indicates no effect. Effect estimates are based on a random-effects model. Each subtotal shows the overall effect estimate for the time-period indicated.

https://doi.org/10.1371/journal.pone.0132340.g004

thumbnail
Fig 5. Forrest plot for total cost of services outcome.

Effect estimates are the standardised mean difference, where the solid vertical line at 0 indicates no effect. Effect estimates are based on a random-effects model. Each subtotal shows the overall effect estimate for the time-period indicated.

https://doi.org/10.1371/journal.pone.0132340.g005

thumbnail
Fig 6. Forrest plot for utilisation of primary and non-specialist care outcome.

Effect estimates are the standardised mean difference, where the solid vertical line at 0 indicates no effect. Effect estimates are based on a random-effects model. Each subtotal shows the overall effect estimate for the time-period indicated.

https://doi.org/10.1371/journal.pone.0132340.g006

thumbnail
Fig 7. Forrest plot for utilisation of secondary care outcome.

Effect estimates are the standardised mean difference, where the solid vertical line at 0 indicates no effect. Effect estimates are based on a random-effects model. Each subtotal shows the overall effect estimate for the time-period indicated.

https://doi.org/10.1371/journal.pone.0132340.g007

thumbnail
Fig 8. Forrest plot for patient satisfaction outcome.

Effect estimates are the standardised mean difference, where the solid vertical line at 0 indicates no effect. Effect estimates are based on a random-effects model. Each subtotal shows the overall effect estimate for the time-period indicated.

https://doi.org/10.1371/journal.pone.0132340.g008

Health.

A statistically significant effect favouring case management was found for self-assessed health status (Fig 3) in the short-term (0.07, 95% CI 0.00 to 0.14, I2 = 35.1%, p = 0.094), but this effect was not present in the long-term (-0.01, 95% CI -0.08 to 0.05, I2 = 12.8%, p = 0.327). No significant effect was found for mortality (short-term: 0.08, 95% CI -0.03 to 0.19, I2 = 63.6%, p = 0.001; long-term: 0.03, 95% CI -0.04 to 0.09, I2 = 40.0%, p = 0.067 –Fig 4).

Cost.

No significant effect was found for total cost of services (short-term: -0.00, 95% CI -0.07 to 0.06, I2 = 0.0%, p = 0.784; long-term: -0.03, 95% CI -0.16 to 0.10, I2 = 46.0%, p = 0.116 –Fig 5), utilisation of primary and non-specialist care (short-term: -0.08, 95% CI -0.22 to 0.05, I2 = 79.2%, p<0.001; long-term: -0.10, 95% CI -0.29 to 0.09, I2 = 78.6%, p<0.001 –Fig 6) or secondary care (short-term: 0.04, 95% CI -0.02 to 0.10, I2 = 39.6%, p = 0.027; long-term: -0.02, 95% CI -0.08 to 0.04, I2 = 22.8%, p = 0.194 –Fig 7).

Satisfaction.

Patient satisfaction (Fig 8) showed a statistically significant beneficial effect in the case management group in the short-term (0.26, 95% CI 0.16 to 0.36, I2 = 0.0%, p = 0.465), increasing in the long-term (0.35, 95% CI 0.04 to 0.66, I2 = 88.3%, p<0.001).

Heterogeneity, measured with the I2 statistic, varied by outcome and time-period measured. Those with particularly high I2 (over 75% [47]), included utilisation of primary and non-specialist care (short- and long-term), and patient satisfaction (long-term).

The funnel plots showed a fairly even distribution of small studies, suggesting no small study bias. The one exception was for self-assessed health status, which appeared slightly skewed towards favourable results for the intervention in smaller studies. However, results of the Egger test found no statistically significant small-study effects across any of the outcomes assessed.

Subgroup analyses

The following outcome categories met the minimum criteria of 10 studies contributing to the primary analysis: mortality (short-, and long-term), self-assessed health status (short-term), utilisation of primary and non-specialist care (short-term), and utilisation of secondary care (short-, and long-term).

The results for each of the subgroup analyses are summarised in Table 7, below (the forest plots can be found in S2 Appendix).

thumbnail
Table 7. Results of subgroup analyses.

No significant differences between subgroups (p<0.05). Note: Positive effect size favours case management for all measures.

https://doi.org/10.1371/journal.pone.0132340.t007

Power to determine differences in subgroup analyses is limited, the large number of comparisons risks inflating rates of Type I error, and there may be other differences between studies that have not been taken into account in these univariate comparisons. Therefore, these results should be treated with appropriate caution. When interpreting subgroup effects, significant difference between subgroups is the important comparative factor. Importantly, no statistically significant differences were found when comparing between subgroups.

However, results perhaps indicate slightly beneficial effects of delivery of case management by an MDT, with the inclusion of a social worker, and in settings with low strength of primary care. These preliminary findings may merit further investigation. Nevertheless, any significant within-subgroup effects found were extremely small by Cohen’s interpretation.

Sensitivity Analysis and Multiple comparisons

Those studies at highest risk of bias reported findings in the short-term (0–12 months) for utilisation of primary and non-specialist care and utilisation of secondary care [75, 86]. Studies using Veteran participants [67, 76, 77, 93, 102], with over 90% males, reported findings in all outcomes and time-periods assessed.

After adjusting for multiple comparisons, excluding these studies showed no significant difference from the results reported above, either for the primary analysis, or between subgroup differences for the subgroup analyses. The results of the sensitivity analysis can be found in S3 Appendix.

After Holm-Bonferroni correction was applied to all results, only two of the statistically significant results held: the finding of a significant effect on patient satisfaction in the short-term (0–12 months) in the primary analysis, and the same outcome measure in the sensitivity analysis (excluding studies with Veteran participants).

Discussion

Summary of the key findings

Case management of ‘at-risk’ patients in primary care has been promoted as a way of reducing health system pressures, and the most recent iteration of the UK GP contract has provided incentives for its delivery. This evidence identified by this review does not provide strong evidence to suggest that case management is an effective way of alleviating pressure on a health system. Total cost of care, and utilisation of secondary care services do not appear to be significantly affected by case management. There may be a significant effect on self-reported health status with case management. However, the magnitude of the benefit is very modest, does not meet conventional criteria even for a ‘small’ effect, and was not significant after adjustment for multiple comparisons. Case management does improve patient satisfaction when compared to usual care. This is a legitimate outcome for a ‘patient-centred’ health care system, but is rarely seen as the primary aim of case management interventions.

Strengths and limitations

Strengths of this study include the use of PRISMA guidelines, pre-specification of subgroups, as well as the broad search strategy. Unfortunately, the broad search impaired our ability to double-screen all studies at every stage, although we did double-screen a proportion at every stage, and our inter-rater reliability was consistently good. We did not include grey literature, due to the generally lower quality of this literature [110]. We found no evidence of small study bias in our included studies.

Assessing complex service-level interventions is difficult, and RCTs may be particularly problematic in the context of patients with multimorbidity [111]. We included the range of intervention study types considered by ‘The Cochrane Effective Practice and Organisation of Care (EPOC) Group’.

We view the use of meta-analysis as a major strength of this piece of work, which differentiates this review from the narrative syntheses [17, 2224, 2628, 30]. Some argue that meta-analysis of complex service-level interventions is inappropriate, because the effects of the intervention are so dependent on context [40], and pooling the results from different contexts is not advisable. However, as shown in the introduction, case management can be defined in terms of a number of common components. In addition, we did try to account for context differences (such as strength of the primary care system), although the precise scope of the term is unclear [112], and a lack of consistent reporting limited what was possible.

Heterogeneity was high for measures of utilisation of primary and non-specialist care in both time-periods assessed, and patient satisfaction in the long-term. This high level of heterogeneity is expected in analysis of a complex intervention, which is possibly highly dependent on context. On the whole, choosing a random effects model took into account expected heterogeneity arising from comparison of a complex intervention across different settings [113]. Nevertheless, caution must be applied to uncritical interpretation of the pooled effect, due to the level of unexplained variation observed.

When we adjusted for multiple comparisons, only increased patient satisfaction in the short-term remained significant. This type of adjustment, while it reduces the risk of false positive findings (type I error), does so at the risk of inflating the number of false negative findings (type II error) [114]. As an intervention with low risk of harm to the patient, we have chosen to present the unadjusted results as the primary analyses, with the results adjusted for multiple comparisons suggesting additional caution in interpretation.

The outcome measures we chose were broadly inclusive. For example, in self-assessed health status we included activities of daily living, as well as bed days, and more typical ‘health’ measures, for instance QALYs. This could be a potential weakness of this study. However, we chose these broad outcome categories attempting to synthesise as much of the relevant data as possible that were reported within the selected studies. Furthermore, these measures were reported as functional outcome measures of health in the individual studies, and were therefore synthesised as such.

Utilisation and cost outcomes have a tendency to be skewed. As expected, the studies we synthesised reporting these outcomes demonstrated significant skew (i.e. the mean is smaller than twice the standard deviation), indicating that the mean reported is not a good indicator of the centre of the distribution [115]. Future primary studies should make sure these skews are reported, and that the effects of any subsequent log transformation are detailed for more precise synthesis of these outcomes. Furthermore, although costs were detailed in a number of studies, we identified only one cost-effectiveness analysis [82], and one cost-benefit analysis [84].

Interpretation of the results in the context of other studies

It is difficult to directly compare, as most previous reviews on this subject have used narrative synthesis methods [17, 23, 24], or used ‘vote counting’ to quantify the number of studies with statistically significant results in either direction [22, 2628, 30]. The majority of existing reviews conclude that despite theoretical benefits, in practice there is only slight evidence of benefits [22, 23], particularly related to patient satisfaction [24, 27], and functional health [26]. The single previous systematic review we identified which employed meta-analysis, additionally included hospital discharge planning interventions (identifying a total of eleven studies, only six of which—three in the primary care setting—were included in meta-analysis), and only used meta-analysis for a single outcome category, ‘unplanned hospital admissions’, similarly finding no significant effect [25]. Our results are in line with those previous reviews, with the additional benefits of updating the evidence base, quantifying the impacts (emphasising the small benefits) across a range of outcome categories, and exploring contextual variations.

Most published reviews focused on implementation have similarly identified inadequate reporting of the methods of applying case management in practice as a major limitation of the literature [17].

Implications for research

Case management has potential to impact on patient safety issues in primary care, such as co-ordination and communication between professionals and levels of the health system [116]. No safety outcomes were identified in the included literature, and primary care patient safety is a notoriously under-researched area [117].

As multimorbid patients are likely at most risk for co-ordination failures, and therefore potentially have most to gain from the integration of care, measures of multimorbidity must be more consistently reported [118], even if this is a simple count of mean number of chronic diseases. Ideally, however, this would give more detailed breakdown by disease type/cluster [119] as a subgroup analysis, enabling further targeting of specific interventions to specific groups of patients who are most likely to benefit [120]. Additionally, there is some evidence that the coexistence of physical and mental health problems could lead to increased management difficulties [121]. Comorbidity of conditions should be better reported in evaluations and explored in further research.

Current evidence comes from a majority of high-income, Western settings. This potential bias requires addressing with evidence from other settings, for example Asia, where the case management approach is currently evolving.

Implications for policy and practice

Given the lack of significant effects across the majority of outcome categories, should case management generally be encouraged or incentivised for the treatment of ‘at-risk’ patients in primary care? This review would suggest that, as currently delivered, case management should not be regarded as a primary means of reducing overall health service utilisation and that it will not reduce costs or improve health outcomes. While we have shown some statistically significant benefits, these are not focused on primary outcomes, with the largest overall effect on satisfaction, which did not meet the usual criterion for a ‘medium’ effect [36]. However, the current results rest on the evidence accumulated from RCTs. There are potential problems associated with this study design in the assessment of complex interventions and conditions [111], although other designs which may be better able to reflect routine delivery of case management (such as controlled before and after designs [80]) have their own problems with internal validity.

Evidence from the subgroup analyses do perhaps point to more effective ways of delivering the intervention, namely: delivery by a MDT as opposed to a single case manager, and the inclusion of a social worker. These findings agree with the wider literature which advocates the use of a multidisciplinary team to successfully manage patients with chronic disease [42], and advocates better integration of health and social care [45]. Case management may be more effective in a system where the strength of primary health care orientation is low. However, these subgroup results should be interpreted with caution, as they are exploratory univariate analyses, which should be investigated further while controlling for potential confounding factors before firm conclusions are drawn. Furthermore, the significance of these effects did not withstand adjustment for multiple comparisons.

Further understanding of factors driving the effectiveness of case management may benefit from on-going evaluation of implementation at the local level. It is important that components of implementation are reported consistently and in detail, so that these can be included in future systematic reviews and effectiveness of individual elements of the intervention can be examined.

Conclusions

Current evidence suggests case management of ‘at-risk’ patients in primary care is not effective beyond small improvements in patient satisfaction. Case management should not be regarded as a proven technology in the delivery of integrated care, there remains a need for further enhancement and evaluation of its effectiveness, particularly with study designs which better incorporate context, and in lower income settings. More research is needed into more effective methods of delivery (e.g. by an MDT and including a social worker), and implementation (e.g. in a health system with poor primary care orientation), which may additionally improve effectiveness. Even with these improvements, however, case management may never be as effective as it needs to be to deliver major savings through a focus on high risk groups [122]. This highlights the need for a variety of models to deal with system pressures, including integrated care at different levels of the health care system, and with more of focus on the wider population of patients [123].

Supporting Information

S2 Appendix. Forest plots for subgroup analyses.

https://doi.org/10.1371/journal.pone.0132340.s002

(DOCX)

S3 Appendix. Results of the sensitivity analyses.

https://doi.org/10.1371/journal.pone.0132340.s003

(DOCX)

Acknowledgments

Dr Søren Rud Kristensen for his feedback and support during the writing-up phase. Levine et al, for supplying the tables from their paper not available in the online version.

Author Contributions

Conceived and designed the experiments: JS PB KC SCS. Performed the experiments: JS MP RA. Analyzed the data: JS MP RA. Wrote the paper: JS MP RA KC SCS PB.

References

  1. 1. Nolte E, McKee M. Caring for people with chronic conditions: a health system perspective: McGraw-Hill International; 2008.
  2. 2. Gröne O, Garcia-Barbero M. Integrated care: a position paper of the WHO European office for integrated health care services. International Journal of Integrated Care. 2001;1.
  3. 3. Kodner DL, Spreeuwenberg C. Integrated care: meaning, logic, applications, and implications—a discussion paper. Int J Integr Care. 2002;2:e12. Epub 2006/08/10. pmid:16896389; PubMed Central PMCID: PMC1480401.
  4. 4. Ham C, Walsh N. Making integrated care happen at scale and pace. London: The King's Fund, 2013.
  5. 5. Atun R, Aydin S, Chakraborty S, Sümer S, Aran M, Gürol I, et al. Universal health coverage in Turkey: enhancement of equity. The Lancet. 2013;382(9886):65–99.
  6. 6. Ross S, Curry N, Goodwin N. Case Management: What it is and how it can best be implemented: King's Fund; 2011.
  7. 7. Powers BW, Chaguturu SK, Ferris TG. Optimizing high-risk care management. Jama. 2015.
  8. 8. Case Management Society of America. Definition of Case Managagement 2007 [cited 2014 19/11/2014]. Available: http://www.cmsa.org.
  9. 9. Goodwin N, Sonola L, Thiel V, Kodner DL. Co-ordinated care for people with complex chronic conditions. London: The King's Fund, 2013.
  10. 10. Georghiou T, Steventon A, Billings J, Blunt I, Lewis G, Bardsley M. Predictive risk and health care: an overview. Evidence for better health care [Internet]. 2011. Available: http://www.nuffieldtrust.org.uk/sites/files/nuffield/publication/Predictive-risk-and-health-care-an-overview_0.pdf.
  11. 11. Lewis G, Curry N, Bardsley M. Choosing a predictive risk model: a guide for commissioners in England. London: Nuffield Trust. 2011.
  12. 12. Starfield B, Shi L, Macinko J. Contribution of Primary Care to Health Systems and Health. The Milbank quarterly. 2005;83(3):457–502. PMC2690145. pmid:16202000
  13. 13. Monitor. Closing the NHS funding gap: how to get better value health care for patients 2013.
  14. 14. Shi L. The Impact of Primary Care: A Focused Review. Scientifica. 2012;2012:22.
  15. 15. van den Akker M, Buntinx F, Metsemakers JFM, Roos S, Knottnerus JA. Multimorbidity in General Practice: Prevalence, Incidence, and Determinants of Co-Occurring Chronic and Recurrent Diseases. Journal of Clinical Epidemiology. 1998;51(5):367–75. pmid:9619963
  16. 16. BMA. General practice contract changes 2014–2015: Unplanned admissions Enhanced Service 2014 [cited 2014 12/11/2014]. Available: http://bma.org.uk/working-for-change/negotiating-for-the-profession/bma-general-practitioners-committee/general-practice-contract/unplanned-admissions-2014.
  17. 17. Challis D, Hughes J, Berzins K, Reilly S, Abell J, Stewart K. Self-care and case management in long-term conditions: the effective management of critical interfaces. Report for the National Institute for Health Research Service Delivery and Organisation programme, accessible at http://www.sdo nihr ac uk/files/project/201-final-report pdf. 2010.
  18. 18. Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. The Lancet. 2012;380(9836):37–43.
  19. 19. Naylor MD, Brooten D, Campbell R, Jacobsen BS, Mezey MD, Pauly MV, et al. Comprehensive discharge planning and home follow-up of hospitalized elders: A randomized clinical trial. Jama. 1999;281(7):613–20. pmid:10029122
  20. 20. Cochrane Collaboration. LMIC Filters: Norwegian Satellite of the Cochrane Effective Practice and Organisation of Care Group; 2012 [cited 2014 20/4/2014]. Available: https://epocoslo.cochrane.org/lmic-filters.
  21. 21. Ovid SP. Expert Searches—Primary Care Filter 2014 [cited 2014 20/4/2014]. Available: http://ovidsp.uk.ovid.com/sp-3.13.1a/ovidweb.cgi.
  22. 22. Eklund K, Wilhelmson K. Outcomes of coordinated and integrated interventions targeting frail elderly people: a systematic review of randomised controlled trials. Health & social care in the community. 2009;17(5):447–58. Epub 2009/02/28. pmid:19245421.
  23. 23. Ferguson JA, Weinberger M. Case management programs in primary care. Journal of General Internal Medicine. 1998;13(2):123–6. pmid:9502373
  24. 24. Hickam DH, Weiss JW, Guise J-M, Buckley D, Motu'apuaka M, Graham E, et al. Outpatient case management for adults with medical illness and complex care needs. 2013.
  25. 25. Huntley AL, Thomas R, Mann M, Huws D, Elwyn G, Paranjothy S, et al. Is case management effective in reducing the risk of unplanned hospital admissions for older people? A systematic review and meta-analysis. Family practice. 2013;30(3):266–75. pmid:23315222
  26. 26. Hutt R, Rosen R, McCauley J. Case-managing Long-term Conditions: What impact does it have in the treatment of older people? London: The King's Fund, 2004.
  27. 27. Latour CH, van der Windt DA, de Jonge P, Riphagen II, de Vos R, Huyse FJ, et al. Nurse-led case management for ambulatory complex patients in general health care: a systematic review. Journal of psychosomatic research. 2007;62(3):385–95. Epub 2007/02/28. pmid:17324690.
  28. 28. Oeseburg B, Wynia K, Middel B, Reijneveld SA. Effects of case management for frail older people or those with chronic illness: a systematic review. Nursing research. 2009;58(3):201–10. Epub 2009/05/19. pmid:19448524.
  29. 29. Smith SM, Soubhi H, Fortin M, Hudon C, O’Dowd T. Managing patients with multimorbidity: systematic review of interventions in primary care and community settings. BMJ: British Medical Journal. 2012;345.
  30. 30. You E, Dunt D, Doyle C, Hsueh A. Effects of case management in community aged care on client and carer outcomes: a systematic review of randomized trials and comparative observational studies. BMC Health Services Research. 2012;12(1):395.
  31. 31. Starfield B, Shi L. Policy relevant determinants of health: an international perspective. Health policy (Amsterdam, Netherlands). 2002;60(3):201–18. Epub 2002/04/20. pmid:11965331.
  32. 32. Effective Practice and Organisation of Care (EPOC). Suggested risk of bias criteria for EPOC reviews Oslo: Norwegian Knowledge Centre for the Health Services; 2014 [cited 2014]. Available: https://epoc.cochrane.org/epoc-specific-resources-review-authors.
  33. 33. Marshall M, Leatherman S, Mattke S. Selecting indicators for the quality of health promotion, prevention and primary care at the health systems level in OECD countries. 2004.
  34. 34. Gebreselassie SG, Gashe FE. A systematic review of effect of prenatal zinc supplementation on birthweight: meta-analysis of 17 randomized controlled trials. Journal of health, population, and nutrition. 2011;29(2):134. pmid:21608422
  35. 35. Kontopantelis E, Reeves D. MetaEasy: A meta-analysis add-in for Microsoft Excel. J Stat Software. 2009;30(7).
  36. 36. Cohen J. Statistical Power Analysis for the Behavioral Sciences: L. Erlbaum Associates; 1988.
  37. 37. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in medicine. 2002;21(11):1539–58. Epub 2002/07/12. pmid:12111919.
  38. 38. Sterne JA, Sutton AJ, Ioannidis JP, Terrin N, Jones DR, Lau J, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002. Epub 2011/07/26. pmid:21784880.
  39. 39. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–34. Epub 1997/10/06. pmid:9310563; PubMed Central PMCID: PMC2127453.
  40. 40. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337.
  41. 41. Borrill CS, Carletta J, Carter A, Dawson JF, Garrod S, Rees A, et al. The effectiveness of health care teams in the National Health Service: University of Aston in Birmingham; 2000.
  42. 42. Wagner EH. The role of patient care teams in chronic disease management. BMJ: British Medical Journal. 2000;320(7234):569–72. PMC1117605. pmid:10688568
  43. 43. Department of Health Social Services and Public Safety. Case management. A position paper. 2005 [12/02/2015]. Available: http://www.dhsspsni.gov.uk/case_management.pdf.
  44. 44. Marengoni A, Angleman S, Melis R, Mangialasche F, Karp A, Garmen A, et al. Aging with multimorbidity: a systematic review of the literature. Ageing research reviews. 2011;10(4):430–9. pmid:21402176
  45. 45. Valentijn PP, Schepman SM, Opheij W, Bruijnzeels MA. Understanding integrated care: a comprehensive conceptual framework based on the integrative functions of primary care. International Journal of Integrated Care. 2013;13.
  46. 46. Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev. 2014;4:MR000034. Epub 2014/05/02. pmid:24782322.
  47. 47. Deeks J, Higgins J, Altman D. Chapter 9–Analysing Data and Undertaking Meta-analyses: Cochrane Handbook for Systematic Reviews of Interventions Version 5.1. 0 [updated March 2011]. Cochrane Handbook for Systematic Reviews of Interventions Version. 2011;5(0).
  48. 48. Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions: Version 4.2.6.2006. Available: http://www.cochrane.org/sites/default/files/uploads/Handbook4.2.6Sep2006.pdf.
  49. 49. Lipsey MW, Wilson DB. Practical Meta-Analysis: SAGE Publications; 2001. 264 p.
  50. 50. StataCorp. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP.; 2013.
  51. 51. Harris R, Bradburn M, Deeks J, Harbord R, Altman D, Sterne J. metan: fixed- and random-effects meta-analysis. Stata Journal. 2008;8(1):3–28.
  52. 52. Sterne JAC, Harbord RM. Funnel plots in meta-analysis. Stata Journal. 2004;4(2):127–41.
  53. 53. Harbord RM, Harris RJ, Sterne JAC. Updated tests for small-study effects in meta-analyses. Stata Journal. 2009;9(2):197–210.
  54. 54. Bender R, Lange S. Adjusting for multiple testing—when and how? Journal of Clinical Epidemiology. 2001;54(4):343–9. pmid:11297884
  55. 55. Holm S. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics. 1979:65–70.
  56. 56. Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009;6(7):e1000097. pmid:19621072
  57. 57. Beland F, Bergman H, Lebel P, Clarfield AM, Tousignant P, Contandriopoulos AP, et al. A system of integrated care for older persons with disabilities in Canada: results from a randomized controlled trial. The journals of gerontology Series A, Biological sciences and medical sciences. 2006;61(4):367–73. Epub 2006/04/14. pmid:16611703.
  58. 58. Beland F, Bergman H, Lebel P, Dallaire L, Fletcher J, Contandriopoulos AP, et al. Integrated services for frail elders (SIPA): a trial of a model for Canada. Canadian journal on aging = La revue canadienne du vieillissement. 2006;25(1):5–42. Epub 2006/06/14. pmid:16770746.
  59. 59. Bernabei R, Landi F, Gambassi G, Sgadari A, Zuccala G, Mor V, et al. Randomised trial of impact of model of integrated care and case management for older people living in the community1998. 1348 p.
  60. 60. Bird S, Noronha M, Sinnott H. An integrated care facilitation model improves quality of life and reduces use of hospital resources by patients with chronic obstructive pulmonary disease and chronic heart failure. Australian journal of primary health. 2010;16(4):326–33. Epub 2010/12/09. pmid:21138701.
  61. 61. Boult C, Reider L, Frey K, Leff B, Boyd CM, Wolff JL, et al. Early effects of "Guided Care" on the quality of health care for multimorbid older persons: a cluster-randomized controlled trial. The journals of gerontology Series A, Biological sciences and medical sciences. 2008;63(3):321–7. pmid:18375882
  62. 62. Leff B, Reider L, Frick KD, Scharfstein DO, Boyd CM, Frey K, et al. Guided care and the cost of complex healthcare: a preliminary report. The American journal of managed care. 2009;15(8):555–9. Epub 2009/08/13. pmid:19670959.
  63. 63. Boyd CM, Reider L, Frey K, Scharfstein D, Leff B, Wolff J, et al. The effects of guided care on the perceived quality of health care for multi-morbid older persons: 18-month outcomes from a cluster-randomized controlled trial. J Gen Intern Med. 2010;25(3):235–42. Epub 2009/12/25. pmid:20033622; PubMed Central PMCID: PMCPmc2839336.
  64. 64. Boult C, Reider L, Leff B, Frick KD, Boyd CM, Wolff JL, et al. The effect of guided care teams on the use of health services: results from a cluster-randomized controlled trial. Archives of internal medicine. 2011;171(5):460–6. Epub 2011/03/16. pmid:21403043.
  65. 65. Boult C, Leff B, Boyd CM, Wolff JL, Marsteller JA, Frick KD, et al. A matched-pair cluster-randomized trial of guided care for high-risk older patients. J Gen Intern Med. 2013;28(5):612–21. Epub 2013/01/12. pmid:23307395; PubMed Central PMCID: PMCPmc3631081.
  66. 66. Boyd ML, Fisher B, Davidson AW, Neilsen CA. Community-based case management for chronically ill older adults. Nursing management. 1996;27(11):31–2. Epub 1996/11/01. pmid:8954445.
  67. 67. Burns R, Nichols LO, Graney MJ, Cloar FT. Impact of continued geriatric outpatient management on health outcomes of older veterans. Archives of internal medicine. 1995;155(12):1313–8. Epub 1995/06/26. pmid:7778963.
  68. 68. Burns R, Nichols LO, Martindale-Adams J, Graney MJ. Interdisciplinary geriatric primary care evaluation and management: two-year outcomes. Journal of the American Geriatrics Society. 2000;48(1):8–13. Epub 2000/01/21. pmid:10642014.
  69. 69. Coburn KD, Marcantonio S, Lazansky R, Keller M, Davis N. Effect of a community-based nursing intervention on mortality in chronically ill older adults: a randomized controlled trial. PLoS medicine. 2012;9(7):e1001265. pmid:22815653
  70. 70. Counsell SR, Callahan CM, Clark DO, Tu W, Buttar AB, Stump TE, et al. Geriatric care management for low-income seniors: a randomized controlled trial. Jama. 2007;298(22):2623–33. Epub 2007/12/13. pmid:18073358.
  71. 71. Counsell SR, Callahan CM, Tu W, Stump TE, Arling GW. Cost analysis of the Geriatric Resources for Assessment and Care of Elders care management intervention. Journal of the American Geriatrics Society. 2009;57(8):1420–6. Epub 2009/08/20. pmid:19691149; PubMed Central PMCID: PMCPmc3874584.
  72. 72. Dalby DM, Sellors JW, Fraser FD, Fraser C, van Ineveld C, Howard M. Effect of preventive home visits by a nurse on the outcomes of frail elderly people in the community: a randomized controlled trial. CMAJ: Canadian Medical Association journal = journal de l'Association medicale canadienne. 2000;162(4):497–500. Epub 2000/03/04. pmid:10701382; PubMed Central PMCID: PMCPmc1231166.
  73. 73. de Stampa M, Vedel I, Buyck JF, Lapointe L, Bergman H, Beland F, et al. Impact on hospital admissions of an integrated primary care model for very frail elderly patients. Archives of gerontology and geriatrics. 2014;58(3):350–5. Epub 2014/02/11. pmid:24508468.
  74. 74. Dorr DA, Wilcox AB, Brunker CP, Burdon RE, Donnelly SM. The effect of technology-supported, multidisease care management on the mortality and hospitalization of seniors. Journal of the American Geriatrics Society. 2008;56(12):2195–202. Epub 2008/12/20. pmid:19093919.
  75. 75. Enguidanos SM, Jamison PM. Moving from tacit knowledge to evidence-based practice: the Kaiser Permanente community partners study. Home health care services quarterly. 2006;25(1–2):13–31. Epub 2006/06/29. pmid:16803736.
  76. 76. Fan VS, Gaziano JM, Lew R, Bourbeau J, Adams SG, Leatherman S, et al. A comprehensive care management program to prevent chronic obstructive pulmonary disease hospitalizations: a randomized, controlled trial. Annals of internal medicine. 2012;156(10):673–83. Epub 2012/05/16. pmid:22586006.
  77. 77. Fitzgerald JF, Smith DM, Martin DK, Freedman JA, Katz BP. A case manager intervention to reduce readmissions. Archives of internal medicine. 1994;154(15):1721–9. Epub 1994/08/08. pmid:8042889.
  78. 78. Fordyce M, Bardole D, Romer L, Soghikian K, Fireman B. Senior Team Assessment and Referral Program—STAR. The Journal of the American Board of Family Practice / American Board of Family Practice. 1997;10(6):398–406. Epub 1998/01/04. pmid:9407480.
  79. 79. Gagnon AJ, Schein C, McVey L, Bergman H. Randomized controlled trial of nurse case management of frail older people. Journal of the American Geriatrics Society. 1999;47(9):1118–24. Epub 1999/09/14. pmid:10484257.
  80. 80. Gravelle H, Dusheiko M, Sheaff R, Sargent P, Boaden R, Pickard S, et al. Impact of case management (Evercare) on frail elderly patients: controlled before and after analysis of quantitative outcome data. BMJ: British Medical Journal. 2007;334(7583):31-. PMC1764106. pmid:17107984
  81. 81. Hogg W, Lemelin J, Dahrouge S, Liddy C, Armstrong CD, Legault F, et al. Randomized controlled trial of anticipatory and preventive multidisciplinary team care: for complex patients in a community-based primary care setting. Canadian family physician Medecin de famille canadien. 2009;55(12):e76–85. Epub 2009/12/17. pmid:20008582; PubMed Central PMCID: PMCPmc2793206.
  82. 82. Gray D, Armstrong CD, Dahrouge S, Hogg W, Zhang W. Cost-effectiveness of Anticipatory and Preventive multidisciplinary Team Care for complex patients: evidence from a randomized controlled trial. Canadian family physician Medecin de famille canadien. 2010;56(1):e20–9. Epub 2010/01/22. pmid:20090057; PubMed Central PMCID: PMCPmc2809192.
  83. 83. Kruse RL, Zweig SC, Nikodim B, LeMaster JW, Coberly JS, Colwill JM. Nurse care coordination of older patients in an academic family medicine clinic: 5-year outcomes. Jcom. 2010;17(5):209–15.
  84. 84. Leung AC-t, Liu C-p, Chow NW-s, Chi I. Cost-benefit analysis of a case management project for the community-dwelling frail elderly in Hong Kong. Journal of Applied Gerontology. 2004;23(1):70–85.
  85. 85. Levine S, Steinman BA, Attaway K, Jung T, Enguidanos S. Home care program for patients at high risk of hospitalization. The American journal of managed care. 2012;18(8):e269–76. Epub 2012/08/30. pmid:22928795.
  86. 86. Martin IR, McNamara D, Sutherland FR, Tilyard MW, Taylor DR. Care plans for acutely deteriorating COPD: a randomized controlled trial. Chronic respiratory disease. 2004;1(4):191–5. Epub 2005/11/12. pmid:16281645.
  87. 87. Metzelthin SF, van Rossum E, de Witte LP, Ambergen AW, Hobma SO, Sipers W, et al. Effectiveness of interdisciplinary primary care approach to reduce disability in community dwelling frail older people: cluster randomised controlled trial2013.
  88. 88. Morishita L, Boult C, Boult L, Smith S, Pacala JT. Satisfaction with outpatient geriatric evaluation and management (GEM). The Gerontologist. 1998;38(3):303–8. Epub 1998/06/26. pmid:9640850.
  89. 89. Boult C, Boult LB, Morishita L, Dowd B, Kane RL, Urdangarin CF. A randomized clinical trial of outpatient geriatric evaluation and management. Journal of the American Geriatrics Society. 2001;49(4):351–9. Epub 2001/05/12. pmid:11347776.
  90. 90. Newcomer R, Maravilla V, Faculjak P, Graves MT. Outcomes of preventive case management among high-risk elderly in three medical groups: a randomized clinical trial. Evaluation & the health professions. 2004;27(4):323–48. Epub 2004/10/20. pmid:15492046.
  91. 91. Ploeg J, Brazil K, Hutchison B, Kaczorowski J, Dalby DM, Goldsmith CH, et al. Effect of preventive primary care outreach on health related quality of life among older adults at risk of functional decline: randomised controlled trial2010.
  92. 92. Ródenas F, Garcés J, Carretero S, Megia MJ. Case management method applied to older adults in the primary care centres in Burjassot (Valencian Region, Spain). Eur J Ageing. 2008;5(1):57–66.
  93. 93. Rubenstein LZ, Alessi CA, Josephson KR, Trinidad Hoyl M, Harker JO, Pietruszka FM. A randomized trial of a screening, case finding, and referral system for older veterans in primary care. Journal of the American Geriatrics Society. 2007;55(2):166–74. Epub 2007/02/17. pmid:17302651.
  94. 94. Schraeder C, Shelton P, Sager M. The effects of a collaborative model of primary care on the mortality and hospital use of community-dwelling older adults. The journals of gerontology Series A, Biological sciences and medical sciences. 2001;56(2):M106–12. Epub 2001/02/24. pmid:11213273.
  95. 95. Schraeder C, Fraser CW, Clark I, Long B, Shelton P, Waldschmidt V, et al. Evaluation of a primary care nurse case management intervention for chronically ill community dwelling older people. Journal of Clinical Nursing. 2008;17(11c):407–17.
  96. 96. Shannon GR, Wilber KH, Allen D. Reductions in costly healthcare service utilization: findings from the Care Advocate Program. Journal of the American Geriatrics Society. 2006;54(7):1102–7. Epub 2006/07/27. pmid:16866682.
  97. 97. Alkema GE, Wilber KH, Shannon GR, Allen D. Reduced Mortality: The Unexpected Impact of a Telephone-Based Care Management Intervention for Older Adults in Managed Care. Health Services Research. 2007;42(4):1632–50. PMC1955273. pmid:17610441
  98. 98. Sledge WH, Brown KE, Levine JM, Fiellin DA, Chawarski M, White WD, et al. A randomized trial of primary intensive care to reduce hospital admissions in patients with high utilization of inpatient services. Disease management: DM. 2006;9(6):328–38. Epub 2006/11/23. pmid:17115880.
  99. 99. Stuck AE, Minder CE, Peter-Wuest I, Gillmann G, Egli C, Kesselring A, et al. A randomized trial of in-home visits for disability prevention in community-dwelling older people at low and high risk for nursing home admission. Archives of internal medicine. 2000;160(7):977–86. Epub 2000/04/13. pmid:10761963.
  100. 100. Sylvia ML, Griswold M, Dunbar L, Boyd CM, Park M, Boult C. Guided care: cost and utilization outcomes in a pilot study. Disease management: DM. 2008;11(1):29–36. Epub 2008/02/19. pmid:18279112.
  101. 101. Boyd CM, Shadmi E, Conwell LJ, Griswold M, Leff B, Brager R, et al. A pilot test of the effect of guided care on the quality of primary care experiences for multimorbid older adults. J Gen Intern Med. 2008;23(5):536–42. Epub 2008/02/13. pmid:18266045; PubMed Central PMCID: PMCPmc2324149.
  102. 102. Toseland RW, O'Donnell JC, Engelhardt JB, Hendler SA, Richie JT, Jue D. Outpatient geriatric evaluation and management. Results of a randomized trial. Medical care. 1996;34(6):624–40. pmid:8656727
  103. 103. Toseland RW, O'Donnell JC, Engelhardt JB, Richie J, Jue D, Banks SM. Outpatient geriatric evaluation and management: is there an investment effect? The Gerontologist. 1997;37(3):324–32. pmid:9203756
  104. 104. Engelhardt JB, Toseland RW, O'Donnell JC, Richie JT, Jue D, Banks S. The effectiveness and efficiency of outpatient geriatric evaluation and management. Journal of the American Geriatrics Society. 1996;44(7):847–56. Epub 1996/07/01. pmid:8675938.
  105. 105. Engelhardt JB, Toseland RW, Gao J, Banks S. Long-Term Effects of Outpatient Geriatric Evaluation and Management on Health Care Utilization, Cost, and Survival. Research on Social Work Practice. 2006;16(1):20–7.
  106. 106. van Hout HPJ, Jansen APD, van Marwijk HWJ, Pronk M, Frijters DF, Nijpels G. Prevention of adverse health trajectories in a vulnerable elderly population through nurse home visits: a randomized controlled trial [ISRCTN05358495]. The journals of gerontology Series A, Biological sciences and medical sciences. 2010;65(7):734–42. pmid:20457579
  107. 107. Macinko J, Starfield B, Shi L. The contribution of primary care systems to health outcomes within Organization for Economic Cooperation and Development (OECD) countries, 1970–1998. Health Serv Res. 2003;38(3):831–65. Epub 2003/06/26. pmid:12822915; PubMed Central PMCID: PMCPmc1360919.
  108. 108. Grant CC, Forrest CB, Starfield B. Primary care and health reform in New Zealand. The New Zealand medical journal. 1997;110(1037):35–9. Epub 1997/02/14. pmid:9066566.
  109. 109. Fry J, Horder J. Primary health care in an international context: Nuffield Provincial Hospitals Trust London; 1994.
  110. 110. McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356(9237):1228–31. Epub 2000/11/10. pmid:11072941.
  111. 111. Maciejewski ML, Bayliss EA. Approaches to comparative effectiveness research in multimorbid populations. Medical care. 2014;52 Suppl 3:S23–30. Epub 2014/02/25. pmid:24561754.
  112. 112. Bate P, Robert G, Fulop N, Øvretveit J, Dixon-Woods M. Perspectives on context. London, UK: 2014.
  113. 113. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7(3):177–88. pmid:3802833
  114. 114. Gelman A, Hill J, Yajima M. Why we (usually) don't have to worry about multiple comparisons. Journal of Research on Educational Effectiveness. 2012;5(2):189–211.
  115. 115. Altman DG, Bland JM. Statistics Notes: Detecting skewness from summary information1996. 1200 p.
  116. 116. Sandars J, Esmail A. The frequency and nature of medical error in primary care: understanding the diversity across studies. Family practice. 2003;20(3):231–6. Epub 2003/05/10. pmid:12738689.
  117. 117. Panagioti M, Stokes J, Esmail A, Coventry P, Cheraghi-Sohi S, Alam R, et al. Multimorbidity and patient safety incidents in primary care: A systematic review and meta-analysis. PLoS ONE. 2015—Submitted.
  118. 118. Kenning C, Coventry PA, Bower P. Self-management interventions in patients with long-term conditions: a structured review of approaches to reporting inclusion, assessment, and outcomes in multimorbidity. 2014. 2014;4(1):9. Epub 2014-07-24.
  119. 119. Islam MM, Valderas JM, Yen L, Dawda P, Jowsey T, McRae IS. Multimorbidity and Comorbidity of Chronic Diseases among the Senior Australians: Prevalence and Patterns. PLoS ONE. 2014;9(1):e83783. pmid:24421905
  120. 120. Fried TR, O’Leary J, Towle V, Goldstein MK, Trentelange M, Martin DK. The Effects of Comorbidity on the Benefits and Harms of Treatment for Chronic Disease: A Systematic Review. PLoS ONE. 2014;9(11):e112593. pmid:25402463
  121. 121. Moussavi S, Chatterji S, Verdes E, Tandon A, Patel V, Ustun B. Depression, chronic diseases, and decrements in health: results from the World Health Surveys. Lancet. 2007;370(9590):851–8. Epub 2007/09/11. pmid:17826170.
  122. 122. Roland M, Abel G. Reducing emergency admissions: are we on the right track? BMJ. 2012;345.
  123. 123. Alderwick H, Ham C, Buck D. Population health systems: Going beyond integrated care. London, UK: The King's Fund, 2015.