Abstract
Background Patients with diabetes have an elevated risk of developing complicated lower respiratory tract infections (LRTIs). However, up until now, GPs have not had the tools to assess individual risks.
Aim To assess the applicability of an existing prediction rule for complicated LRTI among patients with diabetes.
Design of study Retrospective cohort study.
Setting The Utrecht GP Research Network.
Method An existing rule that was used estimates the risk of 30-day hospitalisation or death following an episode of LRTI. Predictors were exacerbation of chronic obstructive pulmonary disease, or pneumonia, increasing age, heart failure, number of hospitalisations in the previous year, use of antibiotics in the previous month, diabetes medication, and prednisone use. Discriminative capacity of the rule was estimated in patients with diabetes. Other potential predictors from the original study were examined, to test for a potentially improved model.
Results Of 445 episodes of LRTI in patients with diabetes, 68 had an outcome of hospitalisation or death within 30 days of diagnosis of LRTI (15.3%). Results showed good reliability of the model (goodness of fit test P = 0.16) and discriminative properties (area under the receiver operating characteristic curve: 0.79, 95% confidence interval = 0.73 to 0.86). No other predictors could be added. Patients with a lower-risk assignment (score ≤2) had a probability of 5.2%, and those with higher risks (score ≥7) had a probability of 36.6% for the composite endpoint of hospitalisation or death within 30 days of diagnosis of LRTI.
Conclusion The use of a prediction rule may help GPs to assess the risk of hospitalisation or death in patients with diabetes who have an episode of LRTI.
INTRODUCTION
Lower respiratory tract infections (LRTIs) are a common reason for people to consult a GP.1,2 Patients with diabetes mellitus have an elevated risk of developing a complicated course of LRTI.3–5 For example, infections may lead to serious acute hyperglycemia, which may cause adverse clinical outcomes.6 Therefore, a careful risk assessment using an accurate, objective prediction rule derived from a primary care population could help GPs to target management of these infections more efficiently in this high-risk group of patients.
Recently, Bont et al published a clinical prediction rule derived from a retrospective cohort study examining 3166 episodes of acute bronchitis, exacerbation of chronic obstructive pulmonary disease (COPD), or pneumonia among an unselected primary care population aged over 65 years.7 The rule estimated the probability of 30-day hospitalisation or death following an episode of LRTI, and the performance of the rule was acceptable (area under the receiver operating characteristic [ROC] curve, or ‘AUC’: 0.75). Although the rule performed similarly in the three separate diagnostic categories (acute bronchitis, and exacerbations of COPD and pneumonia), it is unknown if it can also be applied to the large subgroup of patients at high risk who have diabetes, most of whom have type 2 diabetes. The aim of this study, therefore, was to determine the applicability of the derived clinical prediction rule in older patients with diabetes.
METHOD
This investigation was part of a large retrospective cohort study that was conducted with the use of the computerised medical database of the Utrecht GP Research Network.7 It consisted of retrospectively collected information on a cumulative primary care population of approximately 58 000 patients, from 1995 to 2003. Patient data are registered in records with use of the International Classification of Primary Care (ICPC) codes for diagnoses.8 All prescriptions are recorded in the database, which includes drug name, Anatomical Therapeutic Classification code,9 delivery method, and dose. The system complies with Dutch guidelines on the use of medical data for research purposes and has proved to be valid in pharmaco-epidemiological studies.10
Derivation of the clinical prediction rule among older patients with LRTI
Bont et al obtained medical data from the research database of a retrospective cohort of 1693 older patients aged 65 years and over with a total of 3166 episodes of community-acquired medically attended LRTIs from January 1997 to February 2003.7 LRTI was defined as the occurrence of community-acquired medically attended pneumonia, acute bronchitis, or exacerbations of COPD according to predefined (ICPC) criteria. The composite endpoint for this study was hospitalisation or death within 30 days after diagnosis of LRTI. Patients with more than one episode of LRTI during the study period were also included.
All data were analysed with SPSS (version 12.0). The prior probability of the composite endpoint was 8.7% (274 of 3166 episodes of LRTI).7 Using all data of the 3166 episodes, a multivariable multilevel logistic regression model was developed starting with 20 potential predictors. The following variables appeared to be independent predictors of the composite endpoint: increasing age, hospitalisation in the 12 months prior to diagnosis of LRTI, heart failure, diabetes (indicated by the use of oral diabetes medication or insulin), use of oral glucocorticoids (prednisone), use of antibiotics in the previous month, and a diagnosis of pneumonia or an exacerbation of COPD (all P<0.05). The model was well calibrated (Hosmer–Lemeshow goodness of fit test: P = 0.57), and discriminative properties were acceptable (AUC: 0.75; 95% confidence interval [CI] = 0.72 to 0.78). The prediction rule was derived from the regression coefficients (Table 1). For each episode of LRTI, a sum score was derived by summing the total score on the basis of the pertaining characteristics.
How this fits in
The applicability of a rule that can predict hospitalisation or death in patients who have diabetes and a lower respiratory tract infection (LRTI) has been demonstrated previously. The model comprises easily obtainable clinical characteristics: type of LRTI diagnosis (exacerbation of chronic obstructive pulmonary disease or pneumonia), age, heart failure, hospitalisation in the previous year, use of antibiotics in the previous month, and use of diabetes medication and prednisone. The rule is easy to apply and may help GPs to target preventive and treatment options. This tool improves risk estimation over chance alone. Applying the rule may help GPs to target additional efforts to those patients who need it most.
Application and improvement of the model for patients with diabetes mellitus
To assess the applicability of the developed prediction rule in the total cohort of older patients, the calibration and discriminative properties of the model were quantified for the present study among the subgroup of older persons with insulin-dependent or non-insulin-dependent diabetes. Patients with diabetes (ICPC code T90) were selected from the total cohort of older patients. Interaction between variables included in the model was assessed. Deviations from the additivity assumption were determined by including first-order interaction terms in the model.
Calibration indicates the extent to which the observed frequencies of the composite endpoint agree with the predicted risks according to the prediction rule. The calibration was tested across deciles of the predicted risk with the Hosmer–Lemeshow goodness of fit test.11
Discrimination was estimated with the AUC. This curve illustrates the ability of the model to discriminate between patients with and without the endpoint at subsequent cut-off points along the range of the predicted probabilities. An AUC of 0.5 indicates no discrimination above chance, whereas 1.0 indicates perfect discrimination. Performance of the model in clinical practice was shown by presenting observed risks of the endpoint across the different score classes as defined in the original study.7
Multivariable logistic regression analysis was applied with all potential predictors from the original study to test for a potentially improved model. Finally, calibration was tested again with the Hosmer–Lemeshow goodness of fit test, and discriminative capacity with the AUC.
RESULTS
Of 3166 episodes of LRTI among older persons, 445 were in patients with diabetes. The mean age of patients with diabetes was 76 years (standard deviation 7, range 65–104 years), 55.7% were females, and 21.6% of the patients had been hospitalised at least once in the previous year. Of the patients, 11.7% used insulin, 56.9% used oral diabetes medication, 2.0% used both insulin and oral diabetes medication, and 29.4% did not use any glucose-lowering medication. Frequent comorbid conditions were cardiovascular disease (heart failure: 33.7%, angina pectoris: 25.4%, myocardial infarction: 18.4%, and cerebrovascular disease: 11.9%); emphysema or asthma (45.8%); neurological disease including dementia (10.8%); and renal disease (5.6%).
Exacerbations of COPD were present in 43.4% of the episodes of LRTI, and acute bronchitis and pneumonia were present in 36.6% and 20.0% of the episodes respectively. Within 30 days, 55 (12.4%) of the episodes ended with hospitalisation, and 13 (2.9%) were fatal. The prior probability of a combined endpoint was 15.3%, which was almost twice as high as in the total population of older people. In 57.8% of the episodes the reason for hospitalisation or death was acute respiratory disease, in 25.3% the cause was cardiovascular disease, and in the remaining 17.2% glycaemic deregulation was the most common cause.
Exacerbations of COPD were present in 36.8% of the episodes with an outcome of hospitalisation or death, and acute bronchitis and pneumonia were present in 17.6% and 45.6% of those episodes, respectively (Table 2).
The multivariable logistic regression model, in which the predictors from the original study were entered, showed statistically significant independent associations between seven predictors and the composite endpoint (Table 2). For exacerbation of COPD, no significant association was found (odds ratio: 0.95; 95% CI = 0.41 to 2.19,P = 0.91). Deviations from the additivity assumption were all non-significant. The Hosmer–Lemeshow goodness of fit test indicated that the model was well calibrated (P = 0.16). Discriminative properties of the rule were good (AUC: 0.79, 95% CI = 0.73 to 0.86). Multivariable logistic regression analysis with all potential predictors from the original study did not improve the model.
To gain insight into the practical implications of using the proposed cut-off scores of the prediction rule developed in the original study, the probability of the outcome was shown for different cut-offs in the population of patients with diabetes. Patients with lower-risk assignment (score ≤2) had a probability of 5.2%, and those with higher risk (score ≥7) had a probability of 36.6% for an endpoint (Table 3).
Taking a cut-off score of ≥3 predicts an outcome with a sensitivity of 91.2% and a specificity of 28.9%. A cut-off score of ≥7 predicts an outcome with a sensitivity of 66.2% and specificity of 79.3%. For instance, the prognostic score for an 81-year-old-woman with diabetes and a diagnosis of pneumonia, using insulin is 8 (2 + 2 + 4), which represents a high-risk score. The prognostic score for an 81-year-old-man with diabetes and heart failure using oral diabetes medication is 5 (2 + 2 + 1), which represents a lower-risk score.
DISCUSSION
Summary of main findings
The prediction rule for the probability of hospitalisation or death derived from an unselected population of older people with LRTI appeared to have acceptable discriminative properties in patients with diabetes from the same sample and can be used to target the management of acute bronchitis, and exacerbations of COPD and pneumonia.7 Of 262 patients with a first episode of LRTI, 36 (13.7%) were hospitalised or died within 30 days of LRTI diagnosis.
Strengths and limitations of the study
This study has distinctive strengths. This is the first study that shows individual risks for 30-day hospitalisation or death following an episode of LRTI in patients with diabetes. Although diabetes and the risk of mortality in patients with infections has been described,4,12,13 the present study is the first showing a prognostic score in these patients. In addition, a large range of potential predictors were studied in a primary care setting in which most LRTIs are clinically presented. Finally, the developed prediction rule is easy to apply for all patients with LRTI and performed accurately in this specific high-risk group. The prediction rule allows GPs to avoid laboratory and radiographic tests that are expensive and elaborate to perform in daily practice and may, therefore, lead to a cost-effective preventive and therapeutic management strategy with less patient delay.
The main weakness of the study is the retrospective design; for example, information about glucose levels and type of diabetes was missing. However, it has been shown that risk factors for infection-related mortality did not differ in a subgroup of patients with type 2 diabetes compared with all patients with diabetes.12 It has to be taken into account that most of the patients in the study had type 2 diabetes, as only older people were included and 88% of them did not use insulin. Also, signs and symptoms could not be studied.
Finally, the study population was probably a selection of all patients with diabetes. How this rule performs for patients admitted to hospital is unclear.
Comparison with existing literature
The study data showed that the prior probability for a complicated course of LRTI in the subgroup of patients with diabetes (15.3%) was almost twice as high as in the total population of older people.7 A recent study from the present study group also showed that the risk for LRTI is increased by 46% in type 1 diabetes, and 30% in type 2 diabetes.3 The increase for a recurrent episode of LRTI in type 2 diabetes was even higher (57%).3
Different results have been found regarding age as a possible predictor.7 Also, heart failure has been described as a predictor of infection-related hospitalisation and mortality in patients with and without diabetes.12,14 In this study, exacerbation of COPD did not decrease the discriminative ability of the model. Recent antibiotic use has also been described as a risk factor.15 Studies on LRTI in general found males to be associated with increased mortality in patients with community-acquired pneumonia;16,17 however, in a recent study on the outcome of pneumonia in patients with diabetes this association was not found.5 The present study also did not find such an association.
Implications for future research and clinical practice
The cut-off level could be chosen depending on the acceptability of the proportion of missed outcomes. This study showed a range of cut-off levels, based on those of the existing rule (Table 3). With increasing cutoff scores, the proportion of non-selected persons would increase, but the proportion of outcomes missed would also increase accordingly. Taking a score of ≥7 as the cut-off for patients at high risk, the average probability for a combined endpoint is 37%. Patients at low risk (≤2) may be suitable for home treatment, whereas those at high risk (≥7) might be monitored more closely in a homecare or hospital setting.
A separate rule for patients with diabetes is unhelpful and unlikely. This study showed that a single rule with diabetes as one of the elements in the score and the same cut-off levels could be used in all patients.
In addition to clinical judgement, the prediction rule presented can be used in all patients who have diabetes in primary care who are aged 65 years or older. Implementation of the rule might be facilitated by using computerised medical files with pop-up alerts as reminders. Also, the rule can become part of guidelines and the risk factors can be used without a score chart, making clinical judgements more expedient. For patients at high risk of hospitalisation or death, GPs may make accompanying management decisions, for example, additional monitoring of glucose levels and more-intensive treatment of high-risk comorbid conditions such as heart failure and COPD.
If the discriminative capacity of the rule is confirmed in external populations, future prospective trials should focus on the effectiveness and safety of the rule's application. Moreover, prognostic studies should find out if this prediction rule can also be applied to younger patients with diabetes. Future clinical studies should also demonstrate whether, for example, tachycardia and low blood pressure improve the discrimination of the prediction rule.
The prediction rule to detect risk of hospitalisation or death in an population of older people with LRTI in primary care appeared to have acceptable discriminative properties in older patients with diabetes. Simple variables available when these patients first visit the GP may be used for risk stratification to assess the risk for a complicated course of LRTI. Applying this rule may optimise both preventive and treatment options, and help GPs to target additional efforts to those patients who need it most.
Acknowledgments
We thank the GPs of the University Medical Center Utrecht GP Research Network for supplying the data.
Notes
Funding body
Public Health Fund, The Hague, the Netherlands (grant U03/175-P230). Dutch Diabetes Research Foundation, Amersfoort, the Netherlands (grant 2003.00.031)
Ethical approval
Ethical approval was not necessary
Competing interests
The authors have stated that there are none
Discuss this article
Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss
- Received October 26, 2007.
- Revision received March 11, 2008.
- Accepted April 18, 2008.
- © British Journal of General Practice, 2008.