Abstract
OBJECTIVE: The performance of standard comorbidity scores to control confounding is poorly defined in health care utilization data across elderly populations. We sought to evaluate and rank the performance of comorbidity scores across selected U.S. and Canadian elderly populations using health care utilization databases.
DESIGN: Cross-population validation study.
PARTICIPANTS: Study participants were residents age 65 years or older who had prescription drug coverage through state-funded programs selected from several large health care utilization databases available to the investigators: British Columbia, BC (N=141,161), New Jersey, NJ (N=235,881), and Pennsylvania, PA (N=230,913).
MEASUREMENTS: We calculated 6 commonly used comorbidity scores for all subjects during the baseline year (1994 for NJ and PA, and 1995 for BC). These included scores based on diagnoses (Romano, Deyo, D’Hoore, Ghali) and prescription drugs (CDS-1, CDS-2). The study outcome was 1-year mortality. The performance of scores was measured by c-statistics derived from multivariate logistic regression that included age and gender.
MAIN RESULTS: Across these 4 large elderly populations, we found the same rank order of performance in predicting 1-year mortality after including age and gender in each model: Romano (c-statistic 0.754 to 0.771), Deyo (c-statistic 0.753 to 0.768), D’Hoore (c-statistic 0.745 to 0.760), Ghali (c-statistic 0.733 to 0.745), CDS-1 (c-statistic 0.689 to 0.738), CDS-2 (c-statistic 0.677 to 0.718), and age and gender alone (c-statistic 0.664 to 0.681). Performance was improved by an average of 6% by adding the number of different prescription drugs received during the past year.
CONCLUSIONS: Performance ranking of 6 frequently used comorbidity scores was consistent across selected elderly populations. We recommend that investigators use these performance data as one important factor when selecting a comorbidity score for epidemiologic analyses of health care utilization data.
Similar content being viewed by others
References
Schneeweiss S, Maclure M. Use of comorbidity scores for control of confounding in studies using administrative databases. Int J Epidemiol. 2000;29:891–8.
Greenfield S, Nelson EC. Recent developments and future issues in the use of health status assessment measures in clinical setting. Med Care. 1992;30:MS23-MS41.
Iezzoni LI. Risk Adjustment for Measuring Healthcare Outcomes. 2nd ed. Chicago, Ill: Health Administration Press; 1997.
Roos LL, Sharp SM, Cohen MM, Wajda A. Risk adjustment in claims-based research: the search for efficient approaches. J Clin Epidemiol. 1989;42:1193–1206.
Romano PS, Roos LL, Jollis JG. Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. J Clin Epidemiol. 1993;46:1075–9.
Romano PS, Roos LL, Jollis JG. Further evidence concerning the use of a clinical comorbidity index with ICD-9-CM administrative data. J Clin Epidemiol. 1993;46:1085–90.
von Korff M, Wagner EH, Saunders K. A chronic disease score from automated pharmacy data. J Clin Epidemiol. 1992;45:197–203.
Clark DO, von Korff M, Saunders K, Baluch WM, Simon GE. A chronic disease score with empirically derived weights. Med Care. 1995;33:783–95.
Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–9.
D’Hoore W, Sicotte C, Tilquin C. Risk adjustment in outcome assessment: the Charlson Comorbidity Index. Methods Inf Med. 1993;32:382–7.
D’Hoore W, Bouckaert A, Tilquin C. Practical considerations on the use of the Charlson index with administrative data bases. J Clin Epidemiol. 1996;49:1429–33.
Ghali WA, Hall RE, Rosen AK, Ash AS, Moskowitz MA. Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data. J Clin Epidemiol. 1996;49:273–8.
Melfi C, Holleman E, Arthur D, Katz B. Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data. J Clin Epidemiol. 1995;48:917–26.
Poses RM, Smith WR, McClish DK, Anthony M. Controlling for confounding by indication for treatment. Are administrative data equivalent to clinical data? Med Care. 1995;33:AS36-AS46.
Schneeweiss S, Walker AM, Glynn RJ, Maclure M, Dormuth C, Soumerai SB. Outcomes of reference drug pricing for angiotensin-converting enzyme inhibitors. N Engl J Med. 2002;346:822–9.
Schneeweiss S, Seeger J, Maclure M, Wang P, Avorn J, Glynn RJ. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. Am J Epidemiol. 2001;154:854–64.
Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chron Dis. 1987;40:373–83.
Elixhauser A, Steiner C, Harris R, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36:8–27.
Schneeweiss S, Wang PS, Avorn J, Glynn RJ. Improved comorbidity adjustment for predicting mortality in Medicare populations. Health Serv Res. 2003;38:1103–20.
Zhang JX, Iwashyna TJ, Christakis NA. The performance of different lookback periods and sources of information for Charlson comorbidity adjustment in Medicare claims. Med Care. 1999;37:1128–39.
American Society of Health-System Pharmacists: AHFS Drug Information 96. Bethesda, Md; 1996.
Wang PS, Walker A, Tsuang M, Orav EJ, Levin R, Avorn J. Strategies for improving comorbidity measures based on Medicare and Medicaid claims data. J Clin Epidemiol. 2000;53:571–8.
Yuan Z, Cooper GS, Einstadter D, Cebul RD, Rimm AA. The association between hospital type and mortality and length of stay. Med Care. 2000;38:231–45.
Lessler JT, Harris BSH. Medicaid data as a source for postmarketing surveillance information, final report. Research Triangle Park, NC: Research Triangle Institute; 1984.
Iezzoni LI, Foley SM, Daley J, Hughes J, Fisher ES, Heeren T. Comorbidities, complications, and coding bias: does the number of diagnosis codes matter in predicting in-hospital mortality? JAMA. 1992;267:2197–203.
Fisher ES, Whaley FS, Krushat M, et al. The accuracy of Medicare’s hospital claims data: progress has been made, but problems remain. Am J Public Health. 1992;82:243–8.
Anderson GM, Kerluke KJ, Pulcins IR, Hertzman C, Barer ML. Trends and determinants of prescription drug expenditures in the elderly: data from the British Columbia Pharmacare Program. Inquiry. 1993;30:199–207.
Roos LL, Sharp SM, Cohen MM. Comparing clinical information with claims data: some similarities and differences. J Clin Epidemiol. 1991;44:881–8.
Williams JI, Young W. Inventory of studies on the accuracy of Canadian health administrative databases. Technical Report, Institute for Clinical Evaluative Sciences (ICES); December 1996.
Fowles JB, Lawthers AG, Weiner JP, et al. Agreement between physician’s office records and Medicare part B claims data. Health Care Financ Rev. 1995;16:189–99.
Romano PS, Mark DH. Bias in the coding of hospital discharge data and its implications for quality assessment. Med Care. 1994;32:81–90.
Glynn RJ, Monane M, Gurwitz JH, Choodnovskiy I, Avorn J. Agreement between drug treatment and a discharge diagnosis of diabetes. Am J Epidemiol. 1999;149:541–9.
Ash AS, Shwartz M. Evaluating the performance of risk-adjustment methods: dichotomous outcomes. In: Iezzoni LI, ed. Risk Adjustment for Measuring Healthcare Outcomes. 2nd ed. Chicago, Ill: Health Administration Press; 1997.
Wilson PWF, D’Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837–47.
Hosmer DW, Lemeshow S. Applied Logistic Regression, 2nd ed. New York, NY: Wiley; 2000.
Liebetrau AM. Measures of Association. Quantitative Application in the Social Sciences. Vol. 32. Beverly Hills, Calif: Sage Publications; 1983.
Maclure M, Schneeweiss S. Causation of bias: the episcope. Epidemiology. 2001;12:114–22.
Redelmeier DA, Tan SH, Booth GL. The treatment of unrelated disorders in patients with chronic medical diseases. N Engl J Med. 1998;338:1516–20.
Glynn RJ, Monane M, Gurwitz JH, Choodnovskiy I, Avorn J. Aging, comorbidity, and reduced rates of drug treatment for diabetes mellitus. J Clin Epidemiol. 1999;52:781–90.
Glynn RJ, Knight EL, Levon R, Avorn J. Paradoxical relations of drug treatment with mortality in older persons. Epidemiology. 2001;12:682–9.
Moons KGM, Stijnen T, Michel BC, et al. Application of treatment thresholds to diagnostic test evaluation: an alternative to the comparison of areas under receiver operating characteristics curves. Med Decis Making. 1997;17:447–54.
Hannan EL, Kilburn H, Lindsey ML, Lewis R. Clinical versus administrative data bases for CABG surgery. Does it matter? Med Care. 1992;30:892–907.
Grover S, Coupal L, Hu X-P. Identifying adults at increased risk of coronary disease: how well do the current cholesterol guidelines work? JAMA. 1995;274:801–6.
Wilson PWF, D’Agostino RB Sr., Levy L, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837–47.
Author information
Authors and Affiliations
Corresponding author
Additional information
Supported by grants R01-HS10881 from the U.S. Agency for Healthcare Research and Quality, and R03-AG19463, R03-AG18395, and R01-AG18833 from the National Institute on Aging.
Rights and permissions
About this article
Cite this article
Schneeweiss, S., Wang, P.S., Avorn, J. et al. Consistency of performance ranking of comorbidity adjustment scores in canadian and U.S. utilization data. J GEN INTERN MED 19, 444–450 (2004). https://doi.org/10.1111/j.1525-1497.2004.30109.x
Issue Date:
DOI: https://doi.org/10.1111/j.1525-1497.2004.30109.x