Development of a modified Cambridge Multimorbidity Score for use with SNOMED CT: an observational English primary care sentinel network study

Background People with multiple health conditions are more likely to have poorer health outcomes and greater care and service needs; a reliable measure of multimorbidity would inform management strategies and resource allocation. Aim To develop and validate a modified version of the Cambridge Multimorbidity Score in an extended age range, using clinical terms that are routinely used in electronic health records across the world (Systematized Nomenclature of Medicine — Clinical Terms, SNOMED CT). Design and setting Observational study using diagnosis and prescriptions data from an English primary care sentinel surveillance network between 2014 and 2019. Method In this study new variables describing 37 health conditions were curated and the associations modelled between these and 1-year mortality risk using the Cox proportional hazard model in a development dataset (n = 300 000). Two simplified models were then developed — a 20-condition model as per the original Cambridge Multimorbidity Score and a variable reduction model using backward elimination with Akaike information criterion as the stopping criterion. The results were compared and validated for 1-year mortality in a synchronous validation dataset (n = 150 000), and for 1-year and 5-year mortality in an asynchronous validation dataset (n = 150 000). Results The final variable reduction model retained 21 conditions, and the conditions mostly overlapped with those in the 20-condition model. The model performed similarly to the 37- and 20-condition models, showing high discrimination and good calibration following recalibration. Conclusion This modified version of the Cambridge Multimorbidity Score allows reliable estimation using clinical terms that can be applied internationally across multiple healthcare settings.


INTRODUCTION
Many epidemiological analyses, including measuring the impact of disease or the effectiveness of therapies, require a single measure of comorbidity. People with multiple health conditions ('multimorbidity') are likely to have poorer health outcomes and require more intensive treatment and monitoring, placing significant and increasing demand across the spectrum of health services. 1 Evaluating multimorbidity is important in allocating resources, optimising management strategies, and facilitating research. This can be achieved through composite scores that quantify the effect of specific comorbid conditions on health service use, unplanned hospital admission, and mortality. 2,3 There have been a number of approaches to measuring comorbidity. The Charlson Comorbidity Index (CCI) is a commonly used composite morbidity score with condition weightings based on mortality. 2 However, the management of multimorbidity has seen a paradigm shift towards a greater focus on primary care and non-hospital management of disease; 4-7 the CCI, having been designed for use in secondary care and based on secondary care coding systems, is not ideal for use in primary care. Moreover, the contribution of its 12 selected comorbidities since validation in 1987 has changed, requiring the index to be re-evaluated and re-validated. Other approaches have included the number of comorbidities, although the weakness of this is the lack of weighting, which captures variation in the influence of certain conditions.
To improve on these limitations, the Cambridge Multimorbidity Score (CMMS) was developed in 2020 for use in primary care practices, using data from the Clinical Practice Research Datalink (CPRD). 8 The CMMS used 37 conditions (and 20 in its simplified form) to predict primary care consultations, unplanned hospital admissions, and death as

Abstract Background
People with multiple health conditions are more likely to have poorer health outcomes and greater care and service needs; a reliable measure of multimorbidity would inform management strategies and resource allocation.

Aim
To develop and validate a modified version of the Cambridge Multimorbidity Score in an extended age range, using clinical terms that are routinely used in electronic health records across the world (Systematized Nomenclature of Medicine -Clinical Terms, SNOMED CT).

Design and setting
Observational study using diagnosis and prescriptions data from an English primary care sentinel surveillance network between 2014 and 2019.

Method
In this study new variables describing 37 health conditions were curated and the associations modelled between these and 1-year mortality risk using the Cox proportional hazard model in a development dataset (n = 300 000). Two simplified models were then developed -a 20-condition model as per the original Cambridge Multimorbidity Score and a variable reduction model using backward elimination with Akaike information criterion as the stopping criterion. The results were compared and validated for 1-year mortality in a synchronous validation dataset (n = 150 000), and for 1-year and 5-year mortality in an asynchronous validation dataset (n = 150 000).

Results
The final variable reduction model retained 21 conditions, and the conditions mostly overlapped with those in the 20-condition model. The model performed similarly to the 37-and 20-condition models, showing high discrimination and good calibration following recalibration. primary outcomes. The weighting-based outcome-specific scores of the CMMS are reported to outperform the CCI across all three primary outcomes. However, the original analysis excluded patients aged <21 years, which may limit its validity and utility in studies that include individuals outside of this age range.
The CMMS was originally developed and validated using comorbidities defined with Read clinical terminology, a thesaurus of clinical terms used to record patient findings and procedures in computerised medical records (CMR). 9 Since April 2016 the Read terminology has not been updated. It was then retired from clinical use in English general practice in 2018 and replaced by Systematized Nomenclature of Medicine -Clinical Terms (SNOMED CT), 10 which is used in electronic health records across the world. Potential benefits of SNOMED CT include its comprehensive nature, its capability to be machine processed, its precise collection of clinical terminology, as well as its international implementation.
During the COVID-19 pandemic it was repeatedly reported that comorbid conditions can have a detrimental impact on the severity and prognosis of COVID-19, and individuals with underlying comorbidities are at much higher risk of severe outcomes and mortality. 11 It is therefore imperative that surveillance and epidemiological studies account for such comorbidities. In order to support the surveillance activity of Oxford-Royal College of General Practitioners (RCGP) Research and Surveillance Centre (RSC), it is essential to have a single comorbidity index that can be used in these studies. The current study was conducted to develop and validate a modified version of the CMMS with an extended age range, which is solely based on SNOMED CT, and using routinely collected primary care data from the RSC.

Data source and variables
The study used pseudonymised CMR data from the RSC sentinel network database, which is recruited to be representative of the general population. 12 The UK has registration-based primary care in which each patient registers with a single general practice. All patients who were registered for ≥12 months before the study index date (study start date) and aged ≥16 years at the study index date for each model were included.
The cohort was split into three separate datasets (development set, validation set 1 with synchronous outcome, and validation set 2 with synchronous and asynchronous outcomes) ( Figure 1) using block randomisation in the ratio of 2:1:1.
To minimise the effect of random variation between practices on mortality, the cohort was separated into four subsets using the best linear unbiased estimator from a mixed-effects logistic regression with age (standardised) and sex fixed effects and a practice random effect, before block randomisation (see Supplementary Figure S1). In the current study the authors further applied similar inclusion/exclusion criteria for selecting individuals to those described in the original analysis (see Supplementary Figure S2). 8 The authors then randomly sampled 300 000, 150 000, and 150 000 individuals from the three datasets (development set, validation set 1, and validation set 2, respectively).
The starting variables underlying the conditions used in the original development and validation were carefully curated by the authors, which was based on prior work on the epidemiology of multimorbidity in the UK, 1,13 with the same definitions and/ or prescribing before the index date applied to SNOMED CT rather than to Read v2 (see Supplementary Table S1). The exact same set of 46 starting variables was built using 66 variables within the authors' Themes, Access, Dynamic Data Services library, and the names from the original CMMS were retained. The authors then applied the same logic and combined the anxiety and depression variables as described in Payne et al 8 to yield 37 variables. Age and sex were

How this fits in
Multimorbidity is associated with poorer health outcomes and greater care needs in those with COVID-19, flu, and other infections. A reliable measure of multimorbidity, such as the Cambridge Multimorbidity Score (CMMS), is needed to support infectious disease epidemiology and other research based on routine data that have their key data recorded using SNOMED CT (Systematized Nomenclature of Medicine -Clinical Terms). A modified version of the CMMS was developed and validated using SNOMED CT, which is more widely used internationally than the Read terminology used for the original version and with an extended age range that the CMMS can be applied to. The unadjusted 21-condition model performed similarly to the original CMMS in predicting mortality with excellent discrimination and reasonable calibration. included as covariates, with age censored at 95 years. The focus, for this study, was only on mortality as an outcome measure.
In this study variable curation is only carried out by practising clinicians with either the initial curation or quality assurance carried out by a practising GP. It involves searching for the relevant high-level concept clinical terms, called 'supertypes', within SNOMED CT. Clusters or individual supertypes or subtypes were excluded using a custom-built 'helper tool' within a secure network. The authors recorded the project the variable was curated for, the date, the data curator, and who conducted the quality assurance process and when; all recorded and retained on a secure digital environment. The clinical data curation team met weekly to ensure consistency across the variable-curation process.
The following variables were extracted: pseudonymised practice and patient identifier (ID), sex, date of birth, date of death, dates of registration and deregistration with a general practice, and the 37 conditions.

Statistical analyses
Two time-to-mortality models were constructed using Cox proportional hazards in the development dataset. First, a model with all 37 conditions as binary indicators, with sex, age (in 10 years), and a quadratic age term included as covariates was run. Then a model was run using the 20 conditions that were considered the most important in Payne et al 8 based on effect size, prevalence, and a combination of effect size and prevalence. Finally, variable reduction was conducted by entering all predictors into a model, and then using backward elimination with the Akaike information criterion (AIC) 14 as the stopping criterion with the 'fastbw' function in the rms package.
Model discrimination was evaluated using pseudo-R 2 , Somers' D, and Harrell's C. 15 R 2 is a measure of explained variation in the model. Somers' D quantifies the prognostic separation between observations with high and low predicted risk. Harrell's C is the ratio of concordant pairs of observations to the number of comparable pairs; it estimates the concordance probability that larger predicted risks are associated with lower survival probabilities, when comparing the rankings of a pair of independent observations. Model calibration was assessed using a calibration curve, and recalibration was performed using resampling cross-validation to correct for overfitting with the 'calibrate' function in the rms package. Using model results from the development dataset, then performance of the models was evaluated in two validation datasets with synchronous and asynchronous outcomes (that is, 1-year follow-up in validation set 1, and 1-year and 5-year follow-ups in validation set 2 as visualised in Figure 1). All data preparation and analyses were conducted in R (version 4.1.0), 16 using the following packages: ggplot2 (version 0.9.1), 17  All potentially identifiable data were pseudonymised as close to source as possible and not made available to researchers; data were not extracted for patients who opted out of data sharing. All data are stored and processed at the Oxford-RCGP Clinical Informatics Digital Hub, University of Oxford. This is listed by Health Data Research UK as a trusted research environment and meets the standards of NHS Digital's Data Security and Protection toolkit (organisation code: EE133863-MSD-NDPCHS).

RESULTS
The three datasets were generally comparable in distribution of age, sex, number of conditions, and follow-up time ( Table 1). Individuals in validation set 2 were slightly younger and healthier as a result of the earlier study index date.
The prevalence of the included 37 conditions in the development dataset is presented in Table 2. Both the rates and the rankings show similar patterns to those observed in CPRD. 8 The top 20 conditions by prevalence and by effect size are listed in Supplementary Table S2.
Discrimination of 1-year mortality using the 37-condition model were high in both validation sets 1 and 2 (Harrell's C 0.92 for both models), and discrimination of 5-year mortality was only marginally worse in the validation dataset (Harrell's C 0.91) ( Table 3). Prediction of 1-year and 5-year mortality using the original simplified 20-condition model showed a similar pattern. The current study's reduced model retained 21 conditions, which partly overlapped with those in the 20-condition model (Table 4), and showed similar performance. The model had reasonable calibration, although it was found to underpredict survival at lower risks (<60%). Much of this underprediction was removed in predictions adjusted for overfitting ( Figure 2).

DISCUSSION Summary
In this study, a modified version of the CMMS was developed and validated for use with a wider age range and using SNOMED CT.
The reduced 21-condition model performed similarly to both the full model and the original 20-condition model in predicting mortality with excellent discrimination and reasonable calibration. The performance of the model was also similar to that of the original CMMS. The authors have opted to use the unadjusted 21-condition model as this will maximise its use in studies of different designs, where researchers can apply their own adjustments for age and sex. The authors plan to use this multimorbidity score in their epidemiological studies (including COVID-19 studies), and to make this available to the wider international SNOMED CT community.

Funding
The UK Health Security Agency (UKHSA) is the principal funder of the Oxford-Royal College of General Practitioners (RCGP) Research and Surveillance Centre (RSC). While no specific funding was allocated for this project, a single multimorbidity measure was required by UKHSA. James P Sheppard is funded by the Wellcome Trust/Royal Society via a Sir Henry Dale Fellowship (reference: 211182/Z/18/Z). James P Sheppard also receives funding via a National Institute for Health Research (NIHR) Oxford Biomedical Research Centre Senior Fellowship. This publication presents independent research supported by the NIHR. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care.

Ethical approval
The Cambridge Multimorbidity Score variable was created to support the authors' work with UKHSA, providing a single measure of comorbidity, primarily for vaccine effectiveness studies.

Strengths and limitations
The study used a large, up-to-date, nationally representative cohort, which included all patients aged ≥16 years, and the results were validated using both synchronous and asynchronous datasets. As this study arose from the need for a single comorbidity measure to support the authors' surveillance activity during the pandemic the authors sampled the development and validation datasets only from their cohort and therefore the index was not validated in other independent samples. The analysis was based on SNOMED CT, which is now used across English general practice as well as internationally. The authors believe the results are generalisable to other cohorts and potentially other countries that use similarly coded primary care data. A full list of clinical terms that make up the variables are provided in Supplementary Appendix S1.
The authors' logic model and weightings, for others to use in other databases, are provided in Supplementary Appendix S2. CMMS weights were only derived for mortality and not unplanned hospital admissions or primary care consultations. Mortality tends to be the most commonly used outcome in the development of comorbidity indices in the literature, 26 and it was felt that having only one set of weights would allow it to be easier to apply to, and interpret in, different datasets.
The list of conditions used in this study were exactly as included in the original development and validation of the CMMS by Payne et al, 8 which was based on earlier literature on multimorbidity in primary care. 1,13 These studies did not include other common conditions that might be expected to be included in other multimorbidity indices or that are highly clinically relevant (for example, obesity).

Comparison with existing literature
A number of comorbidity indices and adaptations have been developed in administrative data studies, which are either solely diagnosis based or solely medication based. 26 The current score uses a different approach that combines information from clinical terms as well as prescriptions, and additionally includes a 12-month timeframe in the definition of certain conditions. This allows the severity and/or recency of some conditions (for example, constipation and cancer) to be taken into account in the calculation of the score.
The current study retained a slightly different set of conditions in the reduced model to that in Payne et al. 8 As variable reduction in the current model was based on AIC rather than the combination of effect size and prevalence, the modified score included some less prevalent conditions that are strongly associated with mortality such

Implications for research and practice
With an ageing population, multimorbidity is increasingly prevalent, particularly in developed countries. Multimorbidity is known to be associated with higher healthcare use, higher healthcare costs, greater mortality, as well as higher risk of active patient safety incidents (for example, adverse drug events) or precursors of safety incidents (for example, medication non-adherence). 28 Some earlier studies also suggest particular clusters of multimorbidity, for example, those that involve mental disorders, may be associated with poorer outcomes, 28 and this is also reflected in the modified version of CMMS where a number of mental conditions and substance use disorders are included. A better understanding of multimorbidity and the specific conditions or clusters that lead to poorer outcomes would inform healthcare coordination and delivery, and the development of interventions to improve outcomes in the subgroups of patients at highest risk. As this version of the CMMS was developed based on a population-based cohort, its relevance and generalisability to specialised settings, for example, a mental health service where the patients' demographic and clinical characteristics may be quite different, remains unclear. Future research should explore whether such a measure is useful in guiding care decision making and better organisation of health services in a range of settings and different subgroups of the population.
In conclusion, in this study the development and validation of a modified version of the CMMS for predicting mortality is described. The inclusion of a wider age range may improve the generalisability of the score over the original. As it is based on SNOMED CT rather than Read codes, it is applicable to today's English general practice data and this should also increase its applicability in other contexts. Future research should investigate whether a measure of multimorbidity may be helpful in informing healthcare coordination and delivery, especially across a range of different healthcare settings.

Data
The Oxford-RCGP RSC dataset can be accessed by researchers; approval is on a project-by-project basis (https://orchid. phc.ox.ac.uk/index.php/rcgp-rsc). Ethical approval by an NHS Research Ethics Committee/other appropriate approval is needed before any data release. Researchers wishing to directly analyse patient-level pseudonymised data will be required to complete information governance training and work on the data from the secure servers at the University of Oxford. Patient-level data cannot be taken out of the secure network.

Provenance
Freely submitted; externally peer reviewed.

Competing interests
Simon de Lusignan is Director of the RCGP RSC; he has had grants through his University from AstraZeneca, GSK, Lily, MSD, Sanofi, Seqirus, and Takeda; and has been an advisory board member for AstraZeneca, Sanofi, Seqirus, and Pfizer.