Abstract
Background The CRB-65 score is a clinical prediction rule that grades the severity of community-acquired pneumonia in terms of 30-day mortality.
Aim The study sought to validate CRB-65 and assess its clinical value in community and hospital settings.
Design of study Systematic review and meta-analysis of validation studies of CRB-65.
Method Medline (1966 to June 2009), Embase (1988 to November 2008), British Nursing Index (BNI) and PsychINFO were searched, using a diagnostic accuracy search filter combined with subject-specific terms. The derived (index) rule was used as a predictive model and applied to all validation studies. Comparison was made between the observed and predicted number of deaths stratified by risk group (low, intermediate, and high) and setting of care (community or hospital). Pooled results are presented as risk ratios (RRs) in terms of over-prediction (RR>1) or under-prediction (RR<1) of 30-day mortality.
Results Fourteen validation studies totalling 397 875 patients are included. CRB-65 performs well in hospitalised patients, particularly in those classified as intermediate (RR 0.91, 95% confidence interval [CI] = 0.71 to 1.17) or high risk (RR 1.01, 95% CI = 0.87 to 1.16). In community settings, CRB-65 over-predicts the probability of 30-day mortality across all strata of predicted risk, low (RR 9.41, 95% CI = 1.75 to 50.66), intermediate (RR 4.84, 95% CI = 2.61 to 8.69), and high (RR 1.58, 95% CI = 0.59 to 4.19).
Conclusion CRB-65 performs well in stratifying severity of pneumonia and resultant 30-day mortality in hospital settings. In community settings, CRB-65 appears to over-predict the probability of 30-day mortality across all strata of predicted risk. Caution is needed when applying CRB-65 to patients in general practice.
INTRODUCTION
Community-acquired pneumonia (CAP) has an annual incidence of 5–11 per 1000, accounting for 5–12% of all cases of adult lower respiratory tract infection managed by GPs.1 Appropriate management for patients with suspected CAP must be determined by GPs at initial presentation, particularly with regard to whether or not to manage a patient in the community or refer them to hospital.2 The proportion of adults with CAP who require hospital admission in the UK varies between 22% and 42%.1 This proportion varies in other countries, possibly due to differing structures of primary and secondary healthcare systems.1 Furthermore, of those initially managed in the community, less than 10% subsequently require hospital admission. At the other end of the severity spectrum, between 5% and 10% of patients admitted to hospital require management in an intensive care unit (ICU).1
Severity assessment in patients with suspected CAP plays a key part in planning appropriate management in both community and hospital settings, as the setting of care has an impact on the level of treatment given to the patient as well as the overall costs of treatment.1 The CRB-65 score (Figure 1) is a clinical prediction rule derived to determine the severity of CAP and to aid the physician in arranging appropriate treatment.3 It ranks patients into strata of low, intermediate, or high risk of mortality based on four criteria (new-onset confusion; respiratory rate ≥30/minutes; systolic blood pressure <90 mmHg or diastolic blood pressure ≤60 mmHg; and age ≥65 years), in terms of estimated mortality risk (Figure 1). It is also forms the basis of an algorithm to determine the most appropriate site of care.1 The CRB-65 score is a simplified version of the CURB-65 rule (inclusion of urea estimation: raised >7 mmol/l), and has been advocated for use in community settings as its criteria depend on clinical assessment (history and physical examination) alone.4
How this fits in
The CRB-65 rule grades the severity of community-acquired pneumonia in terms of 30-day mortality risk. Although it is intended for use in the community setting the majority of validation studies are hospital based. This systematic review suggests that the CRB-65 rule performs well in hospital settings but may over-predict the probability of 30-day mortality in community settings, and should be used with caution.
There are specific methodological standards that are applied to clinical prediction rules before their use in clinical practice can be recommended. Ideally, a rule undergoes three steps: derivation, narrow and broad validation, and impact analysis (randomised controlled trial assessing its effectiveness and cost-effectiveness).5–7
The CRB-65 rule has been validated extensively in separate studies, although for a clinical prediction rule specifically intended for community use there have been few validation studies performed wholly in primary care.1,8 As clinical prediction rules may not perform well in practice because of deficiencies in the development methods or because of differences between the original sample and the validation sample, it is good practice to assess their validity across a broad range of eligible patient groups.9 The aim of this study was to perform a comprehensive systematic review and meta-analysis of validation studies of CRB-65, to determine its accuracy in predicting 30-day mortality and assess how well it performs in community and hospital settings.
METHOD
Search strategy
A search strategy was designed for PubMed in order to retrieve all relevant articles. MeSH terms ‘community acquired infection’, ‘respiratory tract infection’, ‘pneumonia’, ‘severity indices’, and ‘prognosis’ were combined, and potential studies were retrieved. This subject-specific search was combined with a methodological filter for clinical prediction rules.10
PubMed was searched from 1966 to June 2009. In addition, Embase, the Cochrane Library, and Medline databases were searched for relevant studies. Google Scholar was used to track references to each selected study, and hand searches of references in the included studies were also performed. Authors of published studies were also contacted for additional data and for knowledge of any unpublished studies.
Selection
Inclusion criteria for the systematic review were as follows: (1) study design – cohort study; (2) patient population – adult patients (≥16 years) with a primary diagnosis of CAP; (3) explanatory variables – CRB-65 score calculated; (4) setting of care – community-based or hospital-based patients; and (5) outcome measure – death within 30 days. Studies that used the same dataset for more than one publication were included once in the meta-analysis.
Validity assessment
Quality assessment was independently performed by two researchers following the methodological standards of McGinn for validation studies of clinical prediction rules.5
Data abstraction
Abstracts of potential studies were independently screened by two researchers and those considered were read fully in duplicate and their suitability for inclusion in the study was independently determined by both researchers. Disagreements were managed by consensus and the involvement of a third researcher.
Statistical methods
The initial derivation study of the CRB-65 rule was used as the predictive model to which all validation studies were compared.3 The number of deaths as predicted by the CRB-65 severity index was compared to the observed number of deaths in each of the validation studies, across the three strata of risk in the CRB-65 rule – low risk (score = 0), intermediate risk (score = 1 or 2), and high risk (score = 3 or 4) (Figure 1). In order to calculate the predicted score, the proportionate mortality estimate from the original derivation study was applied according to the three risk strata – low risk (mortality = 1.2%), intermediate risk (mortality = 8.2%), and high risk (mortality = 31.3%),3 an approach previously used in a community-based validation study.8 The predicted:observed risk ratios for these three risk strata are presented separately for patients who were treated in the community or in hospital. Results are presented as risk ratios (RRs) with 95% confidence intervals (CIs). RR<1 indicated an under-prediction of death by the CRB-65 rule (the observed number of deaths is greater than the predicted number) and RR>1 indicated an over-prediction of death by the CRB-65 rule (the observed number of deaths is less than the predicted number). RR = 1 indicated a perfect calibration between observed and predicted deaths. Assessment of publication bias was also made by means of generating a funnel plot, and statistical testing for publication bias.
Review Manager 5 software from the Cochrane Collaboration was used to perform the analysis, determine heterogeneity, and produce forest plots. RRs were calculated with 95% CIs using the Mantel-Haenszel statistical method. A random-effects analysis was performed, and heterogeneity was described by the I2 statistic.
RESULTS
Description of included studies
The search identified 14 studies with a total of 397 875 patients (Figure 2).8,11–23 Table 1 summarises the characteristics of the included studies. Additional data were provided from three studies,13,15,23 and clarification of included overlapping subjects obtained from two other studies.12,16 Only one study was performed entirely in a primary care setting,8 although two other studies did include data from patients who were treated as outpatients.12,14 The included studies ranged in size from 105 patients to 388 406 patients, with a broadly equal sex balance. Two studies limited the inclusion for age to ≥65 years.8,23 Most studies also had mortality at 30 days as an outcome measure, corresponding directly in timing to the original derivation study.3 However, two studies reported in-hospital mortality and one reported mortality at 42 days.12,23 Thirty-day mortality was obtained from the authors of one of these studies,23 and in-hospital mortality was used as an approximation to 30-day mortality in the other two. The quality-assessment standards of the included studies are summarised in Figure 3. No publication bias exists in the studies (P>0.05) (Appendix 1).
Characteristics of included studies.
Selection of studies for inclusion in the meta-analysis.
Summary diagram of the quality assessment of the included studies.
Descriptive statistics of CRB-65 risk strata
Of the 397 875 patients included in the studies, 66 827 (17%) were classified as low risk (score = 0), 283 902 (71 %) intermediate risk (score = 1 or 2), and 47 146 (12%) high risk (score = 3 or 4) (Table 2). However, the percentage of patients in the three risk strata varied substantially depending on the setting of care, with community-based individuals being categorised into lower-risk categories. Observed events were much less frequent in community-based settings (Table 2).
Descriptive statistics for CRB-65 risk categories and observed risk for hospital and community-based subjects.
Validation of CRB-65 risk strata
For lower-risk individuals, there are no events in community-based patients with consequent over-prediction; in low-risk hospital-based patients there is relative under-prediction, but the 95% CI around this estimate is compatible with accurate prediction (Figure 4). In intermediate-risk groups, over-prediction persists in community-based subjects, and observed 30-day mortality more closely matches predicted mortality for hospital-based patients, with a tighter CI (RR 0.91, 95% 95% CI 0.71 to 1.17) (Figure 5). For high-risk patients, the magnitude of over-prediction attenuates in community-based subjects, while observed mortality matches that predicted by CRB-65 in hospital-based patients (RR 1.01, 95% CI = 0.87 to 1.16) (Figure 6).
Low-risk group (score = 0) predicted and observed deaths in community and hospital settings.
Intermediate-risk group (score = 1 or 2) predicted and observed deaths in community and hospital settings.
High-risk group (score = 3 or 4) predicted and observed deaths in community and hospital settings.
Sensitivity analysis
The study by Ewig et al has a significantly larger number of included participants than all others, and the results of that study dominate the meta-analysis of the hospital-based patients.22 Sensitivity analysis performed excluding this study shows similar results but with broader CIs, with a low-risk-group RR of 1.25 indicating over-prediction of CRB-65 but with a wider CI (0.60 to 2.59). The intermediate-risk group and high-risk group RR estimates remain similar, being RR 0.98, (95% CI = 0.79 to 1.22) and RR 1.03 (95% CI = 0.87 to 1.21) respectively.
DISCUSSION
Summary of main findings
This systematic review shows that application of CRB-65 accurately predicts 30-day mortality in hospitalised patients across three strata of risk. In community settings, CRB-65 appears to over-predict the probability of 30-day mortality across all strata of predicted risk (Figures 4–6).
Strengths and limitations of the study
Pooling of data across 14 separate validation studies enables assessment of the performance of CRB-65 in different clinical settings, addressing validity, generalisability, and precision of the estimates across the three different strata of risk. Several limitations of this study should be considered when interpreting the results. The low event rate, particularly in communitybased studies, makes precise estimates about CRB-65 performance less certain. Observed events (Table 2) are considerably lower than estimated in the original derivation study (Figure 1) on which clinical guidelines are based.3,4 Individual patient data could not be obtained, so it was only possible to estimate predicted risk by applying the original risk estimates to the strata in each of the validation studies, an approach adopted in a community-based validation study.8 However, as CRB-65 relies on unequivocal clinical characteristics, it is unlikely that misclassification of patients occurred. The hospital-based patients category is dominated by one large study that contributed 98% of the total patient dataset;22 however, sensitivity analysis indicates that results are similar when this study is excluded from the meta-analysis. One study only measured patients who suffered from pneumococcal pneumonia,19 but rather than exclude this study, it was included albeit that the event rate is low and is likely to produce an under-prediction of risk. While more studies validating CRB-65 in primary care are needed, it is likely that a large number of patients will need to be recruited to enable precise estimates of performance to be made.9 The outcome of interest – 30-day mortality – is the final clinical endpoint but more frequently occurring intermediate outcomes might be more relevant to clinical care in a primary care setting, such as admission to hospital, clinical deterioration, or re-consultation. Alternatively, quality of life scores or duration of illness may be more indicative of morbidity associated with CAP, and are more likely to be outcomes of more immediate relevance to patients. Lastly, some studies assessed hospital mortality rather than 30-day mortality and this could account for differences in the performance of the rule between different studies.
Comparison with existing literature
CRB-65 is one of several clinical prediction rules that assess the severity of CAP. Other clinical prediction rules that address complications of CAP include the Pneumonia Severity Index (PSI) and CURB-65. The PSI was the first clinical prediction rule derived and validated as a prognostic indicator for CAP; however, its use of 20 different clinical variables makes calculation complex.24 In addition, the inclusion of laboratory values limits its use in primary care settings, since many GPs wil not have immediate access to such investigations. CURB-65 includes the measurement of urea, and this also limits is timeliness and value in primary care.25 The PSI has been compared to the CURB-65 rule in a number of studies,17,29,26–29 and the CPRs have been found to perform similarly.25 These preliminary results indicate that the greater number of variables in the PSI does not significantly strengthen the power of the rule, and therefore their inclusion may only stand as a hindrance to its use.
A number of other clinical prediction rules have been derived with promising initial results, including SCAP,28 SOAR,23 and CORB,13 along with one clinical prediction rule derived entirely in primary care.30,31 The hospital-derived CPRs all contain some or all of confusion, tachypnoea, hypotension, and age in their scoring, and therefore have similarities to CRB-65; however, these rules differ by including other criteria (such as oxygenation) and by the outcomes they predict: development of severe CAP,28 requirement of ventilatory or inotropic support,13 or death.13,23 The primary care-derived clinical prediction rule has different criteria to CRB-65 (diagnosis, age, comorbid conditions, medications, and recent hospitalisation), and the additional outcome measure of hospitalisation along with death. Although preliminary results are promising, these clinical prediction rules have yet to be validated to a great extent, and therefore their use in clinical practice is more uncertain.9 The use of generic sepsis and early-warning scores has also been proposed but preliminary results indicate that they are less effective than pneumonia-specific rules.11
In terms of comparative prognostic value when comparing CURB-65 with CRB-65, it appears that additional urea measurement does not substantially improve the predictive value of CURB-65.12–15,17,20,23 This, along with the simplicity of CRB-65, indicates that it is emerging as potentially the most useful severity score for CAP.
Implications for future research and clinical practice
The inclusion and exclusion criteria for this meta-analysis were broad and reflect those used in the validation studies themselves but also do prove to be a limitation in the analysis. Significant common comorbid illnesses are known to have an impact on severity;2,31–34 their impact was not assessed in the validation studies. Other known predictors of severity such as nursing home residency, recent hospitalisations, and recent antibiotics were also not addressed.2,31,32,34 However, the derivation cohort for CRB-65 specifically included patients with these predictors, and the clinical prediction rule was developed adjusting for these prognostic markers. If included separately in a severity assessment alongside CRB-65, these factors may improve prediction; however, their presence in both the derivation and validation cohorts indicates that the components of CRB-65 have independent predictive characteristics.
Another additional way in which prediction may be improved is with the use of near-patient tests such as C-reactive protein or procalcitonin. These tests have been shown to be an adjunct in the diagnosis of CAP, and preliminary results indicate they may also have predictive value for severity.16,18,35,36 A role for these tests may be found as an additional predictor of severity alongside CRB-65, but it is more likely they could be used as a further discriminator for the appropriate site of care after application of CRB-65.
In the context of clinical management strategies, over half of patients in community settings were classified as low risk, with over 40% classified as intermediate risk (Table 2). Within this group, CRB-65 was found to over-predict the risk of death. However, the event rate level is low and the extent of over-prediction by CRB-65 remains uncertain. It is possible that investigation of other clinical predictors or near-patient tests may improve prediction, but this requires further study. The addition of other risk factors such as comorbidity would also enable different thresholds to be set in community settings, reflecting the lower absolute number of events compared with hospital-based settings (Table 2).
In hospital-based patients, the severity scores were more evenly distributed, with two-thirds in the intermediate-risk group and one-fifth in the high-risk group. As a prognostic tool, CRB-65 appears to be more clinically useful in hospital settings. As clinical medicine is being driven by evidence-based standards of care,37 prognostic rules are likely to become the benchmark against which acceptable standards of care are judged. In this context, as this systematic review shows, it is important that caution is used when extrapolating evidence from studies that have recruited from hospital-based rather than community-based settings. Accurate risk stratification of likely prognosis requires confirmatory validation in similar populations of patients and settings of care.7
Finally, future work with the CRB-65 rule will allow for more robust determination of its usefulness in both hospital and community-based practice. Assessment of the impact of the rule on antibiotic prescribing, especialy in the context of a low-risk patient, will be worthwhile. Also, determining an appropriate threshold for referral from primary care to hospital may be established, since the community-based results indicate that the rule may over-predict mortality in this setting, particularly in the low- and intermediate-risk groups (Figures 4 and 5).
In conclusion, CRB-65 has been found to accurately predict severity of pneumonia and resultant deaths in a hospital setting across al strata of risk. CRB-65 has not been validated sufficiently in primary care settings and preliminary findings suggest over-prediction, so its value as a prognostic indicator in the community remains uncertain.
Funding body
Maggie McNally was supported as an RCSI Research Studentship, Kirsty O'Brien and Borislav D Dimitrov are supported by the HRB Centre for Primary Care Research under grant HRC/2007/1. The funders had no input into the study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.
Competing interests
The authors have stated that there are none.
Discuss this article
Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss
Acknowledgments
We are very grateful to following investigators for providing us with additional unpublished data from their respective studies: Tobias Welte and Stefan Krüger, CAPNETZ Study Group, Hannover Medical School, Germany; Phyo K Myint, University of East Anglia, UK; Kirsty L Buising, University of Melbourne, Australia; and James D Chalmers, Royal Infirmary of Edinburgh, UK.
Appendix 1 Statistical tests for lack of ‘small-study effects’ or publication bias (funnel plot results)
Funnel plot with 95% confidence limits for existence of publication bias in the meta-analysis.
- Received February 11, 2010.
- Revision received March 11, 2010.
- Accepted March 25, 2010.
- © British Journal of General Practice, 2010.
REFERENCES
In this issue
Jump to section
More in this TOC Section
Related Articles
Cited By...
