Skip to main content

Advertisement

Log in

Consistency of performance ranking of comorbidity adjustment scores in canadian and U.S. utilization data

  • Original Articles
  • Published:
Journal of General Internal Medicine Aims and scope Submit manuscript

Abstract

OBJECTIVE: The performance of standard comorbidity scores to control confounding is poorly defined in health care utilization data across elderly populations. We sought to evaluate and rank the performance of comorbidity scores across selected U.S. and Canadian elderly populations using health care utilization databases.

DESIGN: Cross-population validation study.

PARTICIPANTS: Study participants were residents age 65 years or older who had prescription drug coverage through state-funded programs selected from several large health care utilization databases available to the investigators: British Columbia, BC (N=141,161), New Jersey, NJ (N=235,881), and Pennsylvania, PA (N=230,913).

MEASUREMENTS: We calculated 6 commonly used comorbidity scores for all subjects during the baseline year (1994 for NJ and PA, and 1995 for BC). These included scores based on diagnoses (Romano, Deyo, D’Hoore, Ghali) and prescription drugs (CDS-1, CDS-2). The study outcome was 1-year mortality. The performance of scores was measured by c-statistics derived from multivariate logistic regression that included age and gender.

MAIN RESULTS: Across these 4 large elderly populations, we found the same rank order of performance in predicting 1-year mortality after including age and gender in each model: Romano (c-statistic 0.754 to 0.771), Deyo (c-statistic 0.753 to 0.768), D’Hoore (c-statistic 0.745 to 0.760), Ghali (c-statistic 0.733 to 0.745), CDS-1 (c-statistic 0.689 to 0.738), CDS-2 (c-statistic 0.677 to 0.718), and age and gender alone (c-statistic 0.664 to 0.681). Performance was improved by an average of 6% by adding the number of different prescription drugs received during the past year.

CONCLUSIONS: Performance ranking of 6 frequently used comorbidity scores was consistent across selected elderly populations. We recommend that investigators use these performance data as one important factor when selecting a comorbidity score for epidemiologic analyses of health care utilization data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Schneeweiss S, Maclure M. Use of comorbidity scores for control of confounding in studies using administrative databases. Int J Epidemiol. 2000;29:891–8.

    Article  PubMed  CAS  Google Scholar 

  2. Greenfield S, Nelson EC. Recent developments and future issues in the use of health status assessment measures in clinical setting. Med Care. 1992;30:MS23-MS41.

    Article  PubMed  CAS  Google Scholar 

  3. Iezzoni LI. Risk Adjustment for Measuring Healthcare Outcomes. 2nd ed. Chicago, Ill: Health Administration Press; 1997.

    Google Scholar 

  4. Roos LL, Sharp SM, Cohen MM, Wajda A. Risk adjustment in claims-based research: the search for efficient approaches. J Clin Epidemiol. 1989;42:1193–1206.

    Article  PubMed  CAS  Google Scholar 

  5. Romano PS, Roos LL, Jollis JG. Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. J Clin Epidemiol. 1993;46:1075–9.

    Article  PubMed  CAS  Google Scholar 

  6. Romano PS, Roos LL, Jollis JG. Further evidence concerning the use of a clinical comorbidity index with ICD-9-CM administrative data. J Clin Epidemiol. 1993;46:1085–90.

    Article  Google Scholar 

  7. von Korff M, Wagner EH, Saunders K. A chronic disease score from automated pharmacy data. J Clin Epidemiol. 1992;45:197–203.

    Article  Google Scholar 

  8. Clark DO, von Korff M, Saunders K, Baluch WM, Simon GE. A chronic disease score with empirically derived weights. Med Care. 1995;33:783–95.

    Article  PubMed  CAS  Google Scholar 

  9. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–9.

    Article  PubMed  CAS  Google Scholar 

  10. D’Hoore W, Sicotte C, Tilquin C. Risk adjustment in outcome assessment: the Charlson Comorbidity Index. Methods Inf Med. 1993;32:382–7.

    PubMed  CAS  Google Scholar 

  11. D’Hoore W, Bouckaert A, Tilquin C. Practical considerations on the use of the Charlson index with administrative data bases. J Clin Epidemiol. 1996;49:1429–33.

    Article  PubMed  CAS  Google Scholar 

  12. Ghali WA, Hall RE, Rosen AK, Ash AS, Moskowitz MA. Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data. J Clin Epidemiol. 1996;49:273–8.

    Article  PubMed  CAS  Google Scholar 

  13. Melfi C, Holleman E, Arthur D, Katz B. Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data. J Clin Epidemiol. 1995;48:917–26.

    Article  PubMed  CAS  Google Scholar 

  14. Poses RM, Smith WR, McClish DK, Anthony M. Controlling for confounding by indication for treatment. Are administrative data equivalent to clinical data? Med Care. 1995;33:AS36-AS46.

    PubMed  CAS  Google Scholar 

  15. Schneeweiss S, Walker AM, Glynn RJ, Maclure M, Dormuth C, Soumerai SB. Outcomes of reference drug pricing for angiotensin-converting enzyme inhibitors. N Engl J Med. 2002;346:822–9.

    Article  PubMed  Google Scholar 

  16. Schneeweiss S, Seeger J, Maclure M, Wang P, Avorn J, Glynn RJ. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. Am J Epidemiol. 2001;154:854–64.

    Article  PubMed  CAS  Google Scholar 

  17. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chron Dis. 1987;40:373–83.

    Article  PubMed  CAS  Google Scholar 

  18. Elixhauser A, Steiner C, Harris R, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36:8–27.

    Article  PubMed  CAS  Google Scholar 

  19. Schneeweiss S, Wang PS, Avorn J, Glynn RJ. Improved comorbidity adjustment for predicting mortality in Medicare populations. Health Serv Res. 2003;38:1103–20.

    Article  PubMed  Google Scholar 

  20. Zhang JX, Iwashyna TJ, Christakis NA. The performance of different lookback periods and sources of information for Charlson comorbidity adjustment in Medicare claims. Med Care. 1999;37:1128–39.

    Article  PubMed  CAS  Google Scholar 

  21. American Society of Health-System Pharmacists: AHFS Drug Information 96. Bethesda, Md; 1996.

  22. Wang PS, Walker A, Tsuang M, Orav EJ, Levin R, Avorn J. Strategies for improving comorbidity measures based on Medicare and Medicaid claims data. J Clin Epidemiol. 2000;53:571–8.

    Article  PubMed  CAS  Google Scholar 

  23. Yuan Z, Cooper GS, Einstadter D, Cebul RD, Rimm AA. The association between hospital type and mortality and length of stay. Med Care. 2000;38:231–45.

    Article  PubMed  CAS  Google Scholar 

  24. Lessler JT, Harris BSH. Medicaid data as a source for postmarketing surveillance information, final report. Research Triangle Park, NC: Research Triangle Institute; 1984.

    Google Scholar 

  25. Iezzoni LI, Foley SM, Daley J, Hughes J, Fisher ES, Heeren T. Comorbidities, complications, and coding bias: does the number of diagnosis codes matter in predicting in-hospital mortality? JAMA. 1992;267:2197–203.

    Article  PubMed  CAS  Google Scholar 

  26. Fisher ES, Whaley FS, Krushat M, et al. The accuracy of Medicare’s hospital claims data: progress has been made, but problems remain. Am J Public Health. 1992;82:243–8.

    Article  PubMed  CAS  Google Scholar 

  27. Anderson GM, Kerluke KJ, Pulcins IR, Hertzman C, Barer ML. Trends and determinants of prescription drug expenditures in the elderly: data from the British Columbia Pharmacare Program. Inquiry. 1993;30:199–207.

    PubMed  CAS  Google Scholar 

  28. Roos LL, Sharp SM, Cohen MM. Comparing clinical information with claims data: some similarities and differences. J Clin Epidemiol. 1991;44:881–8.

    Article  PubMed  CAS  Google Scholar 

  29. Williams JI, Young W. Inventory of studies on the accuracy of Canadian health administrative databases. Technical Report, Institute for Clinical Evaluative Sciences (ICES); December 1996.

  30. Fowles JB, Lawthers AG, Weiner JP, et al. Agreement between physician’s office records and Medicare part B claims data. Health Care Financ Rev. 1995;16:189–99.

    PubMed  CAS  Google Scholar 

  31. Romano PS, Mark DH. Bias in the coding of hospital discharge data and its implications for quality assessment. Med Care. 1994;32:81–90.

    Article  PubMed  CAS  Google Scholar 

  32. Glynn RJ, Monane M, Gurwitz JH, Choodnovskiy I, Avorn J. Agreement between drug treatment and a discharge diagnosis of diabetes. Am J Epidemiol. 1999;149:541–9.

    PubMed  CAS  Google Scholar 

  33. Ash AS, Shwartz M. Evaluating the performance of risk-adjustment methods: dichotomous outcomes. In: Iezzoni LI, ed. Risk Adjustment for Measuring Healthcare Outcomes. 2nd ed. Chicago, Ill: Health Administration Press; 1997.

    Google Scholar 

  34. Wilson PWF, D’Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837–47.

    PubMed  CAS  Google Scholar 

  35. Hosmer DW, Lemeshow S. Applied Logistic Regression, 2nd ed. New York, NY: Wiley; 2000.

    Google Scholar 

  36. Liebetrau AM. Measures of Association. Quantitative Application in the Social Sciences. Vol. 32. Beverly Hills, Calif: Sage Publications; 1983.

    Google Scholar 

  37. Maclure M, Schneeweiss S. Causation of bias: the episcope. Epidemiology. 2001;12:114–22.

    Article  PubMed  CAS  Google Scholar 

  38. Redelmeier DA, Tan SH, Booth GL. The treatment of unrelated disorders in patients with chronic medical diseases. N Engl J Med. 1998;338:1516–20.

    Article  PubMed  CAS  Google Scholar 

  39. Glynn RJ, Monane M, Gurwitz JH, Choodnovskiy I, Avorn J. Aging, comorbidity, and reduced rates of drug treatment for diabetes mellitus. J Clin Epidemiol. 1999;52:781–90.

    Article  PubMed  CAS  Google Scholar 

  40. Glynn RJ, Knight EL, Levon R, Avorn J. Paradoxical relations of drug treatment with mortality in older persons. Epidemiology. 2001;12:682–9.

    Article  PubMed  CAS  Google Scholar 

  41. Moons KGM, Stijnen T, Michel BC, et al. Application of treatment thresholds to diagnostic test evaluation: an alternative to the comparison of areas under receiver operating characteristics curves. Med Decis Making. 1997;17:447–54.

    Article  PubMed  CAS  Google Scholar 

  42. Hannan EL, Kilburn H, Lindsey ML, Lewis R. Clinical versus administrative data bases for CABG surgery. Does it matter? Med Care. 1992;30:892–907.

    Article  PubMed  CAS  Google Scholar 

  43. Grover S, Coupal L, Hu X-P. Identifying adults at increased risk of coronary disease: how well do the current cholesterol guidelines work? JAMA. 1995;274:801–6.

    Article  PubMed  CAS  Google Scholar 

  44. Wilson PWF, D’Agostino RB Sr., Levy L, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837–47.

    PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sebastian Schneeweiss MD, ScD.

Additional information

Supported by grants R01-HS10881 from the U.S. Agency for Healthcare Research and Quality, and R03-AG19463, R03-AG18395, and R01-AG18833 from the National Institute on Aging.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Schneeweiss, S., Wang, P.S., Avorn, J. et al. Consistency of performance ranking of comorbidity adjustment scores in canadian and U.S. utilization data. J GEN INTERN MED 19, 444–450 (2004). https://doi.org/10.1111/j.1525-1497.2004.30109.x

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1111/j.1525-1497.2004.30109.x

Key words

Navigation