Research

Performance of the SCORE and Globorisk cardiovascular risk prediction models: a prospective cohort study in Dutch general practice

Merle CA Schoofs, Reinier P Akkermans, Wim JC de Grauw, Bianca WM Schalk, Ineke van Dis, Judith Tjin-A-Ton, Erik WMA Bischoff and Marion CJ Biermans
British Journal of General Practice 2023; 73 (726): e24-e33. DOI: https://doi.org/10.3399/BJGP.2021.0726

Abstract

Background GPs frequently use 10-year-risk estimations of cardiovascular disease (CVD) to identify high- risk patients.

Aim To assess the performance of four models for predicting the 10-year risk of CVD in Dutch general practice.

Design and setting Prospective cohort study. Routine data (2009– 2019) was used from 46 Dutch general practices linked to cause of death statistics.

Method The outcome measures were fatal CVD for SCORE and first diagnosis of fatal or non- fatal CVD for SCORE fatal and non-fatal (SCORE- FNF), Globorisk-laboratory, and Globorisk-office. Model performance was assessed by examining discrimination and calibration.

Results The final number of patients for risk prediction was 1981 for SCORE and SCORE-FNF, 3588 for Globorisk-laboratory, and 4399 for Globorisk- office. The observed percentage of events was 18.6% (n = 353) for SCORE- FNF, 6.9% (n = 230) for Globorisk-laboratory, 7.9% (n = 323) for Globorisk-office, and 0.3% (n = 5) for SCORE. The models showed poor discrimination and calibration. The performance of SCORE could not be examined because of the limited number of fatal CVD events. SCORE-FNF, the model that is currently used for risk prediction of fatal plus non-fatal CVD in Dutch general practice, was found to underestimate the risk in all deciles of predicted risks.

Conclusion Wide eligibility criteria and a broad outcome measure contribute to the model applicability in daily practice. The restriction to fatal CVD outcomes of SCORE renders it less usable in routine Dutch general practice. The models seriously underestimate the 10-year risk of fatal plus non-fatal CVD in Dutch general practice. The poor model performance is possibly because of differences between patients that are eligible for risk prediction and the population that was used for model development. In addition, selection of higher-risk patients for CVD risk assessment by GPs may also contribute to the poor model performance.

INTRODUCTION

Cardiovascular disease (CVD) is the leading cause of hospital admissions and deaths worldwide, contributing to over 30% of global deaths.1,2 Treatment based on 10-year-risk estimations of CVD is included in many clinical guidelines for primary prevention of CVD.3–5 GPs play a key role in the prevention of CVD. They use the 10-year-risk estimations to identify patients with a high risk for CVD and to communicate this risk to their patients. After explaining the potential risks and benefits of treatment, they decide with their patients which preventive action will be taken, such as lifestyle interventions or prescribing medication. For this informed shared decision making, it is essential that risk prediction models are accurate.

In the past decades, numerous models that predict 10-year risk of CVD have been developed.6 Two commonly known models are SCORE7 and Globorisk.8 Globorisk has two versions: one based on laboratory measurements (Globorisk-laboratory, henceforth referred to as Globo-lab) and one based on office measurements only, which are measurements that do not require blood tests (Globorisk-office, henceforth referred to as Globo-office).9

In the Netherlands, the guideline on cardiovascular risk management (CVRM) advises GPs to regularly measure all cardiovascular risk factors and assess 10-year risk of CVD in high-risk patients, including patients with a family history of CVD or dyslipidaemia, risk factors such as smoking, obesity, elevated blood pressure or cholesterol levels, and comorbidities such as diabetes mellitus, rheumatoid arthritis, or chronic obstructive pulmonary disease. SCORE for low-risk countries (that is, a SCORE version for low-risk European regions based on mortality statistics) is adopted in this guideline.10 SCORE predicts the 10-year risk of fatal CVD. In addition, the Dutch guideline reports the 10-year risk of fatal and non-fatal CVD, which is called SCORE fatal and non-fatal (SCORE-FNF) throughout this article, where the SCORE risk for low-risk countries is multiplied by coefficients based on data from the Dutch EPIC-NL cohort.10–12 The derivation cohorts of SCORE, SCORE-FNF, and Globorisk are described in Supplementary Tables S1–S3.

Although SCORE and SCORE-FNF are applied in Dutch general practice, they have not been evaluated in this setting. Before a predictive model can be used in a different setting than the one from which it was derived, it should be successfully externally validated in this new setting. In such a domain validation, the potential for differences between the derivation and validation population is large.13

Many prediction models estimating the risk of cardiovascular disease (CVD), such as SCORE and Globorisk, have been developed in cohorts from a general population. Although SCORE is applied in Dutch general practice, it has not been evaluated in this setting. Also, an external validation of the Globorisk models in a Dutch general practice setting is lacking. This study found that SCORE fatal and non- fatal (SCORE- FNF), Globorisk-laboratory, and Globorisk-office underestimate the 10-year risk of fatal plus non-fatal CVD in most of the patients who have their risk assessed by GPs. This study also showed that outcome definitions and eligibility criteria differed among the models, influencing their clinical applicability.

How this fits in

SCORE is evaluated in population-based studies in various countries,6,14,15 including several samples from the general Dutch population.16–19 Although some SCORE studies recruited patients from general practice,20–24 no study evaluated the real-life use of SCORE by GPs in patients selected by the GP for CVRM. Globorisk has been evaluated in some population-based studies,14,25,26 but not in general practice.

The aim of this study was to validate SCORE, SCORE- FNF, Globo-lab, and Globo-office using real- life data from Dutch general practices for patients who were selected for risk assessment by GPs. The applicability and performance of these four models were examined.

METHOD

Design, setting, and selection study population

This study prospectively reviewed data from electronic health records in the general practice database of the Department of Primary and Community Care at the Radboud University Medical Center. Practices (n = 46) were selected that provided data at baseline, that is, 1 January 2009. All practices provided data for at least 10 years. Patients were included if they were registered at one of these practices at baseline. For the evaluation of SCORE and SCORE-FNF, patients aged 40–70 years without a history of diabetes and CVD at baseline were included.10 For the evaluation of Globo-lab and Globo- office, patients aged 40–74 years without a history of stroke or coronary heart disease were included. Patients were included if predictors were measured by GPs between 1 July 2008 and 1 January 2009. This 6-month period was chosen because it is relatively close to baseline and therefore provides a reasonable reliable estimation of the true value of the predictors at baseline. The authors of the current study assumed that patients with these predictor measurements (see Supplementary Table S4) were selected for risk assessment by GPs, which is a valid assumption according to the GPs in the current study’s research team. To determine the smoking status of the patients, data from between 1 January 2008 and 1 January 2019 was used (see Supplementary Box S1 for more details). Patients were included based on the inclusion and exclusion criteria of the prediction models (see Supplementary Table S4), resulting in three study populations. Each participant could be included in more than one study population.

Prediction models

The SCORE and SCORE-FNF models are based on age, sex, systolic blood pressure (SBP), smoking status, total cholesterol/high-density lipoprotein cholesterol ratio, and rheumatoid arthritis.7,10 To calculate the risk of fatal plus non-fatal CVD based on SCORE-FNF, the risks calculated based on SCORE were multiplied by age- and sex- specific multipliers (see Supplementary Box S2).10

Globo-lab uses information on age, sex, SBP, diabetes, smoking status, and total cholesterol. In the Globo-office model, body mass index replaces total cholesterol and diabetes. These Globorisk models were recalibrated using age- and sex-specific mean risk factor levels and CVD rates for the Netherlands.9

CVD outcomes

The CVD outcomes of the models are as follows:

  • SCORE predicts risk of fatal CVD with an atherosclerotic cause, including ischaemic heart disease, stroke, and abdominal aorta aneurysm;

  • SCORE-FNF predicts risk of fatal CVD with an atherosclerotic cause plus hospital admission for myocardial infarction, heart failure, stroke, and peripheral vascular disease; and

  • Globo-lab and Globo-office predict risk on fatal ischaemic heart disease, stroke, or sudden cardiac death, and non-fatal myocardial infarction or stroke (International Classification of Diseases, 10th Revision [ICD-10] codes are presented in Supplementary Table S5).

All non-fatal outcomes were converted into international classification of primary care (ICPC)-1 and ICPC-2 codes, as diagnoses were coded according to the ICPC-1, ICPC-2, or ICD-10 in the present authors’ database.27,28 To determine the cause and date of death of patients who died within the study period, general practice data were linked to cause of death statistics (ICD-10 codes) from Statistics Netherlands based on sex, date of birth, and four-digit postal code.

Statistical analyses

To assess the applicability of the prediction models, the final number of patients and events for each model were assessed. Discrimination was assessed by calculating Harrell’s C statistic.29–31 Calibration was visually assessed using calibration plots. Predicted risks were plotted against the observed risks, where patients were grouped by decile of predicted risk. The observed CVD risk was obtained using 10-year Kaplan– Meier estimates.32 In addition, flexible calibration curves were plotted using three-knot restricted cubic splines.33 Also the Integrated Calibration Index (ICI), E50, and E90 were calculated (see Supplementary Box S3 for more details).

Examination of the performance of SCORE was not possible because of the low number of fatal CVD events (n = 5). Comparison with Statistics Netherlands CVD mortality data and the declining trend in Dutch CVD mortality rates revealed that this was a plausible number of fatal CVD events.34–36

Several analyses were performed to get more insight into the selection of patients for risk assessment by GPs (see Supplementary Box S4 for more details). Statistical analyses were performed using IBM SPSS Statistics (version 25) or R (version 3.6.3).

RESULTS

Study populations

Figure 1 presents a flowchart of the study population selection. In total, 1981 patients were included in the SCORE populations, 3588 in the Globo-lab population, and 4399 in the Globo-office population.

Figure 1.
Figure 1.

Flowchart of the selection of the study population. BMI = body mass index. CVD = cardiovascular disease. DM = diabetes mellitus. Globo-lab = Globorisk-laboratory. Globo- office = Globorisk-office. Ratio = total cholesterol/high-density lipoprotein cholesterol ratio. SBP = systolic blood pressure. SCORE-FNF = SCORE fatal and non-fatal. TC = total cholesterol.

Supplementary Table S6 presents characteristics of patients that could and could not be linked to cause of death statistics. The mean follow-up time was 8.4 (SD 2.9) years for SCORE and 9.0 (SD 2.3) years for Globo-lab and Globo- office. Table 1 presents baseline values on predictors.

View this table:
Table 1.

Characteristics of included patients at baselinea

Applicability

The 10-year observed risks are 18.6% (n = 353) for SCORE-FNF, 6.9% (n = 230) for Globo-lab, and 7.9% (n = 323) for Globo- office population (see Supplementary Table S7). Of these events, only three additional events for SCORE and four for Globo-lab and Globo- office were found by linkage to cause of death data from Statistics Netherlands (see Supplementary Table S8).

Performance of prediction models

The mean predicted CVD risk was 12.2% for SCORE-FNF, 4.8% for Globo-lab, and 7.8% for Globo-office. The C statistic was 0.613 (95% CI = 0.579 to 0.646) for SCORE- FNF, 0.561 (95% CI = 0.522 to 0.600) for Globo- lab, and 0.539 (95% CI = 0.508 to 0.570) for Globo-office. The ICI was 6.9% for SCORE-FNF, 8.8% for Globo-lab, and 8.4% for Globo-office (Table 2). The calibration plots of the three models showed an underestimation in almost all deciles of predicted risk (Figure 2). The models underestimated the risk for patients with lower predicted risks and overestimated the risk for patients with higher predicted risks, but higher predicted risks were less frequent (Figure 3).

View this table:
Table 2.

Observed and predicted 10-year risk of cardiovascular events by Globo-lab, Globo-office, and SCORE-FNFa

Figure 2.
Figure 2.

Calibration plots of predicted versus observed risks of a) SCORE-FNF; b) Globo- lab; and c) Globo- office. Patients were  grouped based on deciles of predicted risk. The diagonal line represents the line of perfect calibration. See Supplementary Table S7 for more details. Globo- lab = Globorisk- laboratory.  Globo- office = Globorisk-office. SCORE-FNF = SCORE  fatal and non-fatal.

Figure 3.
Figure 3.

Calibration plots of predicted versus observed risks of a) SCORE-FNF; b) Globo-lab; and c) Globo-office using restricted cubic splines. The red lines denote the calibration curves. The diagonal line represents the line of perfect calibration. The density function (black line) shows the distribution of predicted risk (right y-axis). Globo-lab = Globorisk-laboratory. Globo-office = Globorisk-office. SCORE-FNF = SCORE non-fatal and fatal.

Selection of patients

No large differences were found in age and sex distribution between the initial general practice population (n = 73 131) and the general Dutch population (see Supplementary Figure S1). Figure 1 shows that GPs measured predictors only in a limited part of the population. More predictor measurements were performed in older patients and in females aged ≥50 years versus males aged ≥50 years (Table 3). The CVD incidence rate in the SCORE-FNF cohort was lower than in the eligible general practice population for SCORE-FNF; 4.16 and 9.57 per 1000 patients per year, respectively (see Supplementary Table S9). The CVD incidence rate in the Globorisk cohort was 8.04 compared with 7.54 per 1000 patients per year in the eligible general practice population for Globorisk (see Supplementary Table S10).

View this table:
Table 3.

Characteristics of patients that could and could not be included in the models based on availability of predictor measurements

The baseline age and SBP in the source population of the SCORE-FNF cohort, in which everyone was measured, was 49 (SD 12) years and 126 (SD 19) mmHg. In the current general practice source population, in which a selection of patients were measured, the baseline age and SBP were 53 (SD 11) years and 142 (SD 16) mmHg (see Supplementary Table S11).

DISCUSSION

Summary

SCORE-FNF, Globo-lab, and Globo-office underestimate the 10-year risk of fatal plus non-fatal CVD in most of the patients who were selected for risk assessment by GPs. Because of the low number of CVD deaths (n = 5) in the SCORE population, it was not possible to assess the performance of SCORE, which predicts 10-year risk of fatal CVD only. This raises questions regarding the applicability of SCORE as only 0.3% of the SCORE population had a fatal CVD event with the pre-specified ICD-10 codes. Considering the final number of patients, the Globorisk models with 3588 and 4399 patients were more widely applicable than the SCORE models with 1981 patients.

The difference in the CVD incidence rate between the EPIC-NL and general practice cohort (see Supplementary Table S9) indicates that the miscalibration of SCORE- FNF is partly because of differences in incidence between the derivation and validation population.

Strengths and limitations

Major strengths of the present study are that the data were linked to cause of death statistics from Statistics Netherlands and the analysis was performed in general practice patients with a 10-year follow-up period, which is a representative population for the real-life setting where CVD risk charts are used. This study also has some limitations that require consideration.

First, medication use before and during the 10-year follow-up period for patients was not taken into account. Patients at increased risk of CVD may have taken medication to lower their CVD risk, which may have influenced the results. Second, general practice data on CVD diagnoses were used instead of data on CVD hospital admissions. In the Netherlands, GPs receive a letter with the diagnoses of their patients after admission to hospital. These diagnoses, especially such serious diagnoses as CVDs, are usually carefully recorded by the GPs, and will therefore be included in the present data. However, the authors might have overestimated the number of CVD events because not all patients with CVD in primary care are referred to the hospital and some CVDs might be incorrectly diagnosed in primary care. This applies, in particular, to peripheral artery disease and heart failure.37,38 However, based on the Dutch clinical guideline39 and clinical expertise in the research team, the authors do not expect that these missed diagnoses or possible outcome misclassification had a large impact on the present results as peripheral artery disease and heart failure occur mostly in older individuals, who were not included in this study.40,41

Third, the assumption that patients with complete measurements on all predictor variables are the patients who were selected by GPs for cardiovascular risk assessment may not hold for patients in the Globo-office population who did not have a cholesterol measurement. Fourth, the 2006 risk assessment standard,42 which was applicable at baseline in 2009, changed in 201143 and 2019.10 The patient population that was selected for risk assessment changed over time (especially the diagnoses rheumatoid arthritis and chronic kidney disease were added as indicators for risk assessment). The present cohort may therefore be less representative for present- day patients. Fifth, the C statistic29–31 in the current study may be biased as it assumes that censoring is random. It is unlikely that the censoring is random, because patients at risk of CVD are likely to be at higher risk of non-CVD death. However, only 1.4%– 2.5% of the population studied died from a non- CVD cause (see Supplementary Table S8). The authors therefore believe that this bias is relatively small.

Comparison with existing literature

The selection of patients may explain why SCORE-FNF, Globo-lab, and Globo- office underestimate the risk of fatal plus non- fatal CVD in the Dutch general practice setting. The models are mostly developed with data from the general population, whereas Dutch GPs apply the models mainly to patients with a suspected cardiovascular risk. This is in contrast with England, where the NHS Health Check programme involves a systematic CVD risk assessment in all individuals aged 40–74 years without existing CVD, diabetes, and other cardiometabolic diseases.44 Obviously, the patients screened by the Dutch GPs belong to a population with a greater rate of disease. Previous research has shown that multimorbidity was related to more GP consultations.45,46 The present study selected patients who had a clinical history available that included information on risk factors that are needed for risk prediction. Therefore, patients in the present study are more likely to represent a more diseased population than model development cohorts (Table 3). Glynn et al in 2008 showed that multimorbidity increases the risk of the development of CVD, which might have contributed to the underestimation that was found in the present study in the general practice population.47

The present results are in line with a study on the performance of SCORE-FNF in general practices in Norfolk in the UK, which found that the risk charts of the previous Dutch GP’s guideline on CVRM seriously underestimate the risk of non- fatal CVD.22 To the authors’ knowledge, no study has evaluated the performance of the Globorisk models in a general practice setting.

Implications for research and practice

The authors suggest a re-estimation of the calibration coefficients using routine, pooled data from various general practice registries. This would improve the generalisability of the prediction model in the general practice setting.13

Besides re-estimation of coefficients, the addition of new predictors may also improve the performance of risk prediction models.13 The low model performances imply that the incorporation of new risk factors may be needed to improve risk prediction. A predictor that is gaining more attention in the field of CVD risk prediction is social deprivation, which is related to CVD risk.48,49 Social deprivation has been incorporated in the QRISK models that predict CVD risk.50–52 These models have been developed using general practice data and have been shown to perform well in a general practice population in the UK.52

In this study it was not possible to assess the performance of SCORE, as only five fatal CVD events were observed. Although CVD death is the most robust clinical outcome measure, it is less relevant if it occurs so seldomly in this timeframe, with these selection criteria, and these pre- specified ICD-10 codes. Considering the low applicability, it could be questioned whether SCORE should be used at all in the Dutch general practice setting. The SCORE- FNF and Globorisk models are clinically more relevant than SCORE as they include both CVD mortality and CVD morbidity as outcome measures. When compared with the Globorisk models, SCORE-FNF showed a greater applicability regarding the outcome definition. This is mainly because of the inclusion of heart failure and peripheral vascular disease in the outcome definition, which resulted in more events being observed in this population than in the Globorisk population. Another factor that contributes to the model applicability includes the eligibility criteria. The authors found in the present data that about twice as many patients could be included for risk prediction using the Globorisk models than using the SCORE models. Rather than only looking at the predictive performance of the models, the authors think that more attention should be paid to the model applicability, which covers CVD outcome definition and eligibility criteria.

In conclusion, this study found that SCORE-FNF, Globo-lab, and Globo-office underestimate the 10-year risk of fatal plus non-fatal CVD in most of the patients that were selected for risk assessment by GPs. Two reasons may contribute to this. First, differences in CVD incidence between derivation and validation population may contribute to the miscalibration of SCORE- FNF. Second, GPs probably select patients with more disease for CVD risk prediction than patients from the general population that were incorporated in the development cohorts of these models. Re-estimation of coefficients using general practice data and incorporating new risk factors in prediction models may improve risk prediction in a general practice setting. The authors believe that deciding which prediction model should be used for CVD risk prediction in general practice should not only rely on model performance but also on the applicability of the models, which includes broad eligibility criteria and a clinically relevant and frequently occurring outcome definition.

Acknowledgments

The authors would like to thank all participating practices for providing the data; José Donkers, Hans Peters, and Michael Ricking for constructing the dataset; and Goodarz Danaei and Peter Ueda for their contribution in the starting phase of this study and for sharing the Globorisk codes.

Notes

Funding

This work was supported by a grant from the Stoffels-Hornstra Foundation, Coevorden, the Netherlands.

Ethical approval

In the Netherlands, no approval from an ethical committee is necessary for analysing anonymised data from general practitioner databases. All methods were performed in accordance with the relevant guidelines and regulations (Declaration of Helsinki). All GPs gave permission to extract data from their electronic medical records for research purposes and informed their patients who could object to the use of their data. Data extraction from the medical records occurred de-identified. The Radboudumc Medical Ethics Review Committee approved this procedure (reference: 2020-6871). The procedure adheres to the regulations of Dutch and European laws.

Data

The datasets used during the present study are available from the corresponding author on reasonable request.

Provenance

Freely submitted; externally peer reviewed.

Competing interests

The authors have declared no competing interests.

Discuss this article

Contribute and read comments about this article: bjgp.org/letters

  • Received December 30, 2021.
  • Revision requested January 31, 2022.
  • Accepted August 11, 2022.

REFERENCES

  1. 1.↵
    1. GBD 2017 Cause of Death Collaborators
    (2018) Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 392, 10159, 1736–1788.
    OpenUrlCrossRefPubMed
  2. 2.↵
    1. Dagenais GR,
    2. Leong DP,
    3. Rangarajan S,
    4. et al.
    (2020) Variations in common diseases, hospital admissions, and deaths in middle-aged adults in 21 countries from five continents (PURE): a prospective cohort study. Lancet 395, 10226, 785–794.
    OpenUrlPubMed
  3. 3.↵
    1. Arnett DK,
    2. Blumenthal RS,
    3. Albert MA,
    4. et al.
    (2019) 2019 ACC/AHA Guideline on the primary prevention of cardiovascular disease: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation 140, 11, e596–e646.
    OpenUrlCrossRefPubMed
  4. 4.
    1. National Institute for Health and Care Excellence
    (2016) Cardiovascular disease: risk assessment and reduction, including lipid modification CG181, https://www.nice.org.uk/guidance/cg181 (accessed 28 Oct 2022).
  5. 5.↵
    1. Visseren FLJ,
    2. Mach F,
    3. Smulders YM,
    4. et al.
    (2021) 2021 ESC Guidelines on cardiovascular disease prevention in clinical practice. Eur Heart J 42, 34, 3227–3237.
    OpenUrlCrossRefPubMed
  6. 6.↵
    1. Damen JA,
    2. Hooft L,
    3. Schuit E,
    4. et al.
    (2016) Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ 353, i2416.
    OpenUrlAbstract/FREE Full Text
  7. 7.↵
    1. Conroy RM,
    2. Pyörälä K,
    3. Fitzgerald AP,
    4. et al.
    (2003) Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J 24, 11, 987–1003.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Hajifathalian K,
    2. Ueda P,
    3. Lu Y,
    4. et al.
    (2015) A novel risk score to predict cardiovascular disease risk in national populations (Globorisk): a pooled analysis of prospective cohorts and health examination surveys. Lancet Diabetes Endocrinol 3, 5, 339–355.
    OpenUrl
  9. 9.↵
    1. Ueda P,
    2. Woodward M,
    3. Lu Y,
    4. et al.
    (2017) Laboratory-based and office-based risk scores and charts to predict 10-year risk of cardiovascular disease in 182 countries: a pooled analysis of prospective cohorts and health surveys. Lancet Diabetes Endocrinol 5, 3, 196–213.
    OpenUrl
  10. 10.↵
    1. Nederlands Huisartsen Genootschap
    (2019) [Guideline of the Dutch College of General Practitioners: cardiovascular risk management (M84) (version 4.0)]. [Article in Dutch]. https://richtlijnen.nhg.org/standaarden/cardiovasculair-risicomanagement (accessed 28 Oct 2022).
  11. 11.
    1. van Dis I,
    2. Geleijnse JM,
    3. Boer JM,
    4. et al.
    (2014) Effect of including nonfatal events in cardiovascular risk estimation, illustrated with data from the Netherlands. Eur J Prev Cardiol 21, 3, 377–383.
    OpenUrlCrossRefPubMed
  12. 12.↵
    1. Beulens JW,
    2. Monninkhof EM,
    3. Verschuren WM,
    4. et al.
    (2010) Cohort profile: the EPIC-NL study. Int J Epidemiol 39, 5, 1170–1178.
    OpenUrlCrossRefPubMed
  13. 13.↵
    1. Toll DB,
    2. Janssen KJ,
    3. Vergouwe Y,
    4. Moons KG
    (2008) Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol 61, 11, 1085–1094.
    OpenUrlCrossRefPubMed
  14. 14.↵
    1. Fahimfar N,
    2. Fotouhi A,
    3. Mansournia MA,
    4. et al.
    (2022) Prediction of cardiovascular disease mortality in a Middle Eastern country: performance of the Globorisk and Score functions in four population-based cohort studies of Iran. Int J Health Policy Manag 11, 2, 210–217.
    OpenUrl
  15. 15.↵
    1. Neuhauser HK,
    2. Ellert U,
    3. Kurth BM
    (2005) A comparison of Framingham and SCORE-based cardiovascular risk estimates in participants of the German National Health Interview and Examination Survey 1998. Eur J Cardiovasc Prev Rehabil 12, 5, 442–450.
    OpenUrlPubMed
  16. 16.↵
    1. Merry AH,
    2. Boer JM,
    3. Schouten LJ,
    4. et al.
    (2012) Risk prediction of incident coronary heart disease in the Netherlands: re-estimation and improvement of the SCORE risk function. Eur J Prev Cardiol 19, 4, 840–848.
    OpenUrlCrossRefPubMed
  17. 17.
    1. Scheltens T,
    2. Verschuren WM,
    3. Boshuizen HC,
    4. et al.
    (2008) Estimation of cardiovascular risk: a comparison between the Framingham and the SCORE model in people under 60 years of age. Eur J Cardiovasc Prev Rehabil 15, 5, 562–566.
    OpenUrlCrossRefPubMed
  18. 18.
    1. van der Heijden AA,
    2. Ortegon MM,
    3. Niessen LW,
    4. et al.
    (2009) Prediction of coronary heart disease risk in a general, pre-diabetic, and diabetic population during 10 years of follow-up: accuracy of the Framingham, SCORE, and UKPDS risk functions: the Hoorn Study. Diabetes Care 32, 11, 2094–2098.
    OpenUrlAbstract/FREE Full Text
  19. 19.↵
    1. van Dis I,
    2. Kromhout D,
    3. Geleijnse JM,
    4. et al.
    (2010) Evaluation of cardiovascular risk predicted by different SCORE equations: the Netherlands as an example. Eur J Cardiovasc Prev Rehabil 17, 2, 244–249.
    OpenUrlCrossRefPubMed
  20. 20.↵
    1. Barroso LC,
    2. Muro EC,
    3. Herrera ND,
    4. et al.
    (2010) Performance of the Framingham and SCORE cardiovascular risk prediction functions in a non-diabetic population of a Spanish health care centre: a validation study. Scand J Prim Health Care 28, 4, 242–248.
    OpenUrlCrossRefPubMed
  21. 21.
    1. Comín E,
    2. Solanas P,
    3. Cabezas C,
    4. et al.
    (2007) Estimating cardiovascular risk in Spain using different algorithms. Rev Esp Cardiol 60, 7, 693–702.
    OpenUrlCrossRefPubMed
  22. 22.↵
    1. Jørstad HT,
    2. Boekholdt SM,
    3. Wareham NJ,
    4. et al.
    (2017) The Dutch SCORE-based risk charts seriously underestimate the risk of cardiovascular disease. Neth Heart J 25, 3, 173–180.
    OpenUrl
  23. 23.
    1. Jørstad HT,
    2. Colkesen EB,
    3. Minneboo M,
    4. et al.
    (2015) The Systematic COronary Risk Evaluation (SCORE) in a large UK population: 10-year follow-up in the EPIC-Norfolk prospective population study. Eur J Prev Cardiol 22, 1, 119–126.
    OpenUrlCrossRefPubMed
  24. 24.↵
    1. Ramsay SE,
    2. Morris RW,
    3. Whincup PH,
    4. et al.
    (2011) Prediction of coronary heart disease risk by Framingham and SCORE risk assessments varies by socioeconomic position: results from a study in British men. Eur J Cardiovasc Prev Rehabil 18, 2, 186–193.
    OpenUrlCrossRefPubMed
  25. 25.↵
    1. Barr ELM,
    2. Barzi F,
    3. Rohit A,
    4. et al.
    (2020) Performance of cardiovascular risk prediction equations in Indigenous Australians. Heart 106, 16, 1252–1260.
    OpenUrlAbstract/FREE Full Text
  26. 26.↵
    1. Geldsetzer P,
    2. Manne-Goehler J,
    3. Theilmann M,
    4. et al.
    (2018) Geographic and sociodemographic variation of cardiovascular disease risk in India: a cross-sectional study of 797,540 adults. PLoS Med 15, 6, e1002581.
    OpenUrlPubMed
  27. 27.↵
    1. Lamberts H,
    2. Wood M
    (1987) International Classification of Primary Care (Oxford University Press, Oxford).
  28. 28.↵
    1. World Health Organization (WHO)
    (2020) International Classification of Diseases (WHO, Geneva), 10th Revision.
  29. 29.↵
    1. Harrell FE,
    2. Califf RM,
    3. Pryor DB,
    4. et al.
    (1982) Evaluating the yield of medical tests. JAMA 247, 18, 2543–2546.
    OpenUrlCrossRefPubMed
  30. 30.
    1. Harrell Jr FE,
    2. Lee KL,
    3. Califf RM,
    4. et al.
    (1984) Regression modelling strategies for improved prognostic prediction. Stat Med 3, 2, 143–152.
    OpenUrlCrossRefPubMed
  31. 31.↵
    1. Harrell FE Jr.,
    2. Lee KL,
    3. Mark DB
    (1996) Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15, 4, 361–387.
    OpenUrlCrossRefPubMed
  32. 32.↵
    1. Kaplan EL,
    2. Meier P
    (1958) Nonparametric estimation from incomplete observations. JASA 53, 282, 457–481.
    OpenUrlCrossRef
  33. 33.↵
    1. Austin PC,
    2. Harrell FE Jr.,
    3. van Klaveren D
    (2020) Graphical calibration curves and the integrated calibration index (ICI) for survival models. Stat Med 39, 21, 2714–2742.
    OpenUrlCrossRefPubMed
  34. 34.↵
    1. Statline CBS
    (2022) [Deceased; important causes of death (short list), age, sex]. [Article in Dutch]. https://www.cbs.nl/nl-nl/cijfers/detail/7052_95 (accessed 28 Oct 2022).
  35. 35.
    1. Vaartjes I,
    2. O’Flaherty M,
    3. Grobbee D,
    4. et al.
    (2011) Coronary heart disease mortality trends in the Netherlands 1972–2007. Heart 97, 7, 569–573.
    OpenUrlAbstract/FREE Full Text
  36. 36.↵
    1. VZInfo
    (2022) [Trend in mortality due to cardiovascular diseases]. [Article in Dutch]. https://www.volksgezondheidenzorg.info/onderwerp/hart-en-vaatziekten/cijfers-context/trends#node-trend-sterfte-door-hart-en-vaatziekten (accessed 28 Oct 2022).
  37. 37.↵
    1. Valk MJ,
    2. Mosterd A,
    3. Broekhuizen BD,
    4. et al.
    (2016) Overdiagnosis of heart failure in primary care: a cross-sectional study. Br J Gen Pract, DOI: https://doi.org/10.3399/bjgp16X685705.
  38. 38.↵
    1. Kyle D,
    2. Boylan L,
    3. Wilson L,
    4. et al.
    (2020) Accuracy of peripheral arterial disease registers in UK general practice: case-control study. J Prim Care Community Health 11, 2150132720946148.
    OpenUrl
  39. 39.↵
    1. Nederlands Huisartsen Genootschap
    (2019) [NHG-Guideline adequate creation of patient record with the electronic health record system]. [Article in Dutch]. https://www.nhg.org/sites/default/files/content/nhg_org/uploads/final_adepd_2019_web.pdf (accessed 28 Oct 2022).
  40. 40.↵
    1. Bartelink M,
    2. Stoffers H,
    3. Boutens E,
    4. et al.
    (2009) [NHG-standard peripheral arterial disease] [Article in Dutch] (Bohn Stafleu van Loghum, Houten), 213–229.
  41. 41.↵
    1. Bosch L,
    2. Assmann P,
    3. de Grauw WJ,
    4. et al.
    (2019) Heart failure in primary care: prevalence related to age and comorbidity. Prim Health Care Res Dev 20, e79.
    OpenUrl
  42. 42.↵
    1. Nederlands Huisartsen Genootschap
    (2006) [Multidisciplinary guideline of the Dutch College of General Practitioners: cardiovascular risk management]. [Article in Dutch]. https://www.zorgstandaarddiabetes.nl/wp-content/uploads/2013/04/rl_cvrm_2006.pdf(accessed 17 Nov 2022).
  43. 43.↵
    1. Nederlands Huisartsen Genootschap
    (2011) [Multidisciplinary guideline of the Dutch College of General Practitioners: cardiovascular risk management, revision 2011]. [Article in Dutch], https://www.nvvc.nl/Richtlijnen/2011_MDR_CVRM.pdf (accessed 28 Oct 2022).
  44. 44.↵
    1. Artac M,
    2. Dalton A,
    3. Babu H,
    4. et al.
    (2013) Primary care and population factors associated with NHS Health Check coverage: a national cross-sectional study. J Public Health 35, 3, 431–439.
    OpenUrlCrossRefPubMed
  45. 45.↵
    1. van Oostrom SH,
    2. Picavet HSJ,
    3. de Bruin SR,
    4. et al.
    (2014) Multimorbidity of chronic diseases and health care utilization in general practice. BMC Fam Pract 15, 61.
    OpenUrlCrossRefPubMed
  46. 46.↵
    1. Cassell A,
    2. Edwards D,
    3. Harshfield A,
    4. et al.
    (2018) The epidemiology of multimorbidity in primary care: a retrospective cohort study. Br J Gen Pract, DOI: https://doi.org/10.3399/bjgp18X695465.
  47. 47.↵
    1. Glynn LG,
    2. Buckley B,
    3. Reddan D,
    4. et al.
    (2008) Multimorbidity and risk among patients with established cardiovascular disease: a cohort study. Br J Gen Pract, DOI: https://doi.org/10.3399/bjgp08X319459.
  48. 48.↵
    1. Tunstall-Pedoe H,
    2. Woodward M
    (2006) By neglecting deprivation, cardiovascular risk scoring will exacerbate social gradients in disease. Heart 92, 3, 307–310.
    OpenUrlAbstract/FREE Full Text
  49. 49.↵
    1. Brindle P,
    2. Beswick A,
    3. Fahey T,
    4. Ebrahim S
    (2006) Accuracy and impact of risk assessment in the primary prevention of cardiovascular disease: a systematic review. Heart 92, 12, 1752–1759.
    OpenUrlAbstract/FREE Full Text
  50. 50.↵
    1. Hippisley-Cox J,
    2. Coupland C,
    3. Brindle P
    (2017) Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study. BMJ 357, j2099.
    OpenUrlAbstract/FREE Full Text
  51. 51.
    1. Hippisley-Cox J,
    2. Coupland C,
    3. Vinogradova Y,
    4. et al.
    (2007) Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ 335, 7611, 136.
    OpenUrlAbstract/FREE Full Text
  52. 52.↵
    1. Hippisley-Cox J,
    2. Coupland C,
    3. Vinogradova Y,
    4. et al.
    (2008) Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ 336, 7659, 1475–1482.
    OpenUrlAbstract/FREE Full Text
Back to top

In this issue

British Journal of General Practice: 73 (726)
British Journal of General Practice
Vol. 73, Issue 726
January 2023
Download PDF
Download PowerPoint
Article Alerts
Email Article
Citation Tools
Performance of the SCORE and Globorisk cardiovascular risk prediction models: a prospective cohort study in Dutch general practice
Merle CA Schoofs, Reinier P Akkermans, Wim JC de Grauw, Bianca WM Schalk, Ineke van Dis, Judith Tjin-A-Ton, Erik WMA Bischoff, Marion CJ Biermans
British Journal of General Practice 2023; 73 (726): e24-e33. DOI: 10.3399/BJGP.2021.0726
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo

Jump to section

Keywords

Intended for Healthcare Professionals

 
 

British Journal of General Practice