Abstract
Background Population-based screening for cardiovascular disease (CVD) risk, incorporating blood tests, is proposed in several countries.
Aim The aim of this study was to evaluate whether a simple approach to identifying individuals at high risk of CVD using routine data might be effective.
Design of study Prospective cohort study (EPIC-Norfolk).
Setting Norfolk area, UK.
Method A total of 21 867 men and women aged 40–74 years, who were free from CVD and diabetes at baseline, participated in the study. The discrimination (the area under the receiver operating characteristic curve [aROC]), calibration, sensitivity/specificity, and positive/negative predictive value were evaluated for different risk thresholds of the Framingham risk equations and the Cambridge diabetes risk score (as an example of a simple risk score using routine data from electronic general practice records).
Results During 203 664 person-years of follow-up, 2213 participants developed a first CVD event (10.9 per 1000 person-years). The Cambridge diabetes risk score predicted CVD events reasonably well (aROC 0.72; 95% confidence interval [CI] = 0.71 to 0.73), while the Framingham risk score had the best predictive ability (aROC 0.77; 95% CI = 0.76 to 0.78). The Framingham risk score overestimated risk of developing CVD in this representative British population by 60%.
Conclusion A risk score incorporating routinely available data from GP records performed reasonably well at predicting CVD events. This suggests that it might be more efficient to use routine data as the first stage in a stepwise population screening programme to identify people at high risk of developing CVD before more time- and resource-consuming tests are used.
INTRODUCTION
Cardiovascular disease (CVD) is a major cause of death and disability in developed countries.1 Identification of individuals who are at high risk of developing CVD using multivariable risk assessment tools is a recommended approach to primary prevention in many countries.2
In the UK, in line with the Joint British Guidelines,3 the UK National Screening Committee recently recommended that all adults aged 40–74 years who have not previously been identified through self-assessment or record-based screening, should be invited for cardiovascular risk assessment using a Framingham-derived risk score. This score requires blood-pressure measurement and that blood tests are taken to determine an individual's cholesterol level.4 This policy, which will place huge demands on primary care, is being introduced despite considerable uncertainty regarding which methods should be used to identify those at high risk of developing CVD.5,6
The Framingham risk score has previously been shown to overestimate CVD risk in the UK population,7 and it requires blood testing. An alternative approach might be to use non-invasive, routinely available data to stratify the population according to CVD risk and, then, only invite those at highest risk for further assessment. This might reduce the economic costs and potential psychological harms, such as anxiety and false reassurance, associated with screening tests.8
How this fits in
The NHS Health Check programme recommends that all adults aged 40–74 years who have never been identified through self-assessment or record-based screening, should be invited for cardiovascular disease (CVD) risk assessment using a Framingham-derived risk score, which requires individuals to attend their surgery for blood pressure measurement and biochemical testing. The Cambridge diabetes risk score was developed to identify people at high risk of undiagnosed type 2 diabetes, using routinely collected data from general practice records. It has never been assessed for the prediction of CVD. A risk score incorporating routinely available data from GP records performed reasonably well at predicting CVD events. This suggests that it might be more efficient to use routine data as a first step in a stepwise population screening programme to identify people at high CVD risk before more time- and resource-consuming tests are used.
As diabetes and CVD share many common risk factors,9 scores assessing the risk of diabetes may also identify those at risk of developing CVD. The Cambridge diabetes risk score is one example of a tool designed to identify people at high risk of undiagnosed type 2 diabetes, using routinely collected data from electronic general practice records.10 This simple risk score has also been shown to predict prevalent undiagnosed diabetes in different ethnic groups,11 incident diabetes,12 and all-cause mortality,13 but has never been assessed for the prediction of CVD. As such, this study seeks to examine the performance of the Framingham risk equations and the Cambridge risk score (as an example of a simple tool using routinely available data) in estimating CVD risk in a population-based UK cohort.
METHOD
Study design and population
The European Prospective Investigation of Cancer-Norfolk (EPIC-Norfolk) is a population-based prospective study that follows 25 639 men and women aged 40–74 years residing in the Norfolk region of the UK. Details of the study are described elsewhere14 but, in short, between 1993 and 1997, out of 77 630 individuals who were invited to participate in the study via general practice, 25 639 (33%) consented and attended a baseline health assessment.
Participants were questioned about their personal and family history of disease, medication, and lifestyle factors, including smoking habits. They were asked whether a physician had ever told them that they had any of the conditions in a list that included diabetes, heart attack, and stroke. Anthropometric and blood-pressure measurements, as well as non-fasting blood samples, were also taken. Individuals who had CVD at baseline (n = 1106) and those with missing values for one or more of the variables used to calculate the Framingham risk score and the Cambridge risk score (n = 2666) were excluded; this left a total of 21 867 individuals for the main analyses.
The inhabitants of the Norfolk area are slightly healthier than the general UK population, with a standardised mortality ratio of 93.15 However, EPIC-Norfolk is similar to a nationally representative sample regarding anthropometric indices, blood pressure, and serum lipids.14
Follow-up and ascertainment of CVD events
Participants who were free from CVD at the time of recruitment were followed-up for the development of a first CVD event or death. Results for follow-up to 30 April 2007, a median of 11.0 years, are reported.
Incident CVD was defined as a composite of fatal or non-fatal CVD, including hospitalisation from coronary heart disease and stroke, or death from coronary heart disease, stroke, and peripheral vascular disease. Vital status for all EPIC-Norfolk participants was obtained via death certification at the Office for National Statistics. Participants admitted to a hospital were identified by their NHS number. Hospitals were linked to the East Norfolk Health Authority database, which identifies all hospital contacts throughout England and Wales for Norfolk residents. Follow-up data were 95% complete for hospital records and over 99% for mortality. Previous validation studies in this cohort indicated high specificity of such case ascertainment.16
Estimation of cardiovascular risk
The Framingham risk score and Cambridge diabetes risk score were calculated for each individual in the cohort. Table 1 summarises variables used to calculate each score. The Framingham score was derived using the modified Framingham risk functions recently updated by D'Agostino et al,17 which define CVD as the first event of coronary heart disease, cerebrovascular disease, peripheral artery disease (including intermittent claudication), and heart failure. The variables used to calculate the Cambridge score are increasingly routinely available in electronic clinical records in primary care.
Table 1 Variables used to calculate the Framingham risk score and the Cambridge diabetes risk score.
Equations for the QRISK score have not been available in the public domain, and so it was not possible to compare QRISK scores with the Framingham and Cambridge scores.
Statistical analyses
Baseline characteristics were summarised separately for men and women, who did and did not develop a CVD event, using percentages, means, and medians for categorical, normally, and, non-normally distributed data respectively. Differences between groups were tested for using χ2 tests for categorical variables, and t-tests or Kruskal-Wallis tests for normally or non-normally distributed continuous variables.
CVD event rates were calculated by dividing the number of CVD events by person-years of follow-up. Follow-up was defined as the period from the date of first health assessment to the event date (date of hospitalisation or date of death), or 30 April 2007. CVD event rates were calculated during 10 years of follow-up in groups defined by quintiles of the Framingham score and the Cambridge score.
To compare the discrimination of the Framingham score and the Cambridge score, receiver operating characteristic curves (ROCs) were plotted and the areas under the ROC curves (aROCs) were compared using a nonparametric method.18
The calibration of the Framingham score was assessed by comparing the mean predicted to observed 10-year CVD event risk for each quintile of the score using the modified Hosmer-Lemeshow χ2 statistic.19 For this, all analyses were truncated to obtain observed CVD event rates at 10 years using the Kaplan-Meier technique. It was not possible to assess the calibration of the Cambridge score because its values do not represent a 10-year absolute risk of CVD.
The sensitivity, specificity, positive/negative predictive value, and positive clinical likelihood ratios for different risk thresholds of the risk scores were also estimated. The difference in these measures between CVD risk thresholds was tested using McNemar's test for paired proportions. Finally, a sensitivity analysis was conducted to examine whether the predictive ability of the Cambridge score might change after excluding family history of diabetes, smoking, and body mass index (BMI).
RESULTS
Description of study population and CVD event rates
Baseline characteristics of the study population are summarised in Table 2. The mean age of participants at baseline was 58.3 years (standard deviation [SD] = 9.3), and 9602 (44%) were male.
Table 2 Baseline characteristics of the study population.
Individuals who developed CVD were, on average, 7 years older compared with those who did not. They were also more likely to be male and to have a higher BMI, higher systolic blood pressure, and diabetes. People who developed CVD also had higher levels of total cholesterol, triglyceride and HbA1c, and lower HDL-cholesterol values at baseline. The median values of the Framingham score and Cambridge score in those who did develop CVD were higher than those in individuals who did not (Framingham score: 0.26 versus 0.12, P<0.001, Cambridge score: 0.29 versus 0.10, P<0.001). In total, 6640 participants (30.4%) had a predicted 10-year CVD event risk of ≥20% according to the Framingham risk equations.
During 203 664 person-years of follow-up, there were 2213 first CVD events (Table 3). These included 1543 hospitalisations with a diagnosis of coronary heart disease, and 342 with a diagnosis of stroke. There were 218, 46, and 64 deaths from coronary heart disease, stroke, and peripheral vascular disease, respectively. The cumulative incidence rate of CVD was 10.9 per 1000 person-years.
Table 3 Rates of cardiovascular disease events during 10 years of follow-up in the EPIC-Norfolk cohort, stratified by quintile of the Framingham risk equations and the Cambridge diabetes risk score (n = 21 867).
CVD event rates increased significantly with each increasing quintile of the risk scores (Table 3). Individuals in the top quintile of the Framingham score and Cambridge score had a 33 and 15 times higher risk of developing CVD events respectively, compared with those in the lowest quintile.
Predictive ability of the Framingham score and Cambridge score
The aROCs for the Framingham score and the Cambridge score are shown in Figure 1. The Framingham score was significantly better than the Cambridge score at discriminating between individuals who developed a CVD event and those who did not (aROC 0.77 [95% confidence interval (CI) = 0.76 to 0.78], and 0.72 [95% CI = 0.71 to 0.73] for the Framingham score and Cambridge score respectively, P<0.001). The differences in the discriminative ability of the two approaches remained significant when analyses were performed separately for men and women.
Figure 1 Area under the receiver operating characteristic curve (aROC) for the Framingham risk score (FRS) and the Cambridge diabetes risk score (CRS) for the prediction of CVD events in the EPIC-Norfolk cohort (n = 21 867).
Test of calibration
The Framingham score overestimated CVD event risk by 60% (predicted CVD risk = 16.2% and observed CVD risk = 10.1%), with the largest absolute difference of 13.5% in the top quintile of the risk score (Figure 2). The Hosmer-Lemeshow χ2 statistic was 202.5 for the Framingham score, indicating poor goodness of fit (P<0.001).
Figure 2 Predicted versus observed 10-year risk of CVD events by quintile of the Framingham risk score in the EPIC-Norfolk cohort (n = 21 867).
Sensitivity/specificity and positive/negative predictive value
The sensitivity, specificity, positive/negative predictive value, and positive likelihood ratio at different risk thresholds for the Framingham score and Cambridge score are shown in Table 4. About a third of the population had a Framingham score value of ≥0.20, signifying a ≥20% 10-year risk of a CVD event. A Framingham score value of ≥0.20 identified 66% of individuals who developed a first CVD event during 10 years of follow-up (sensitivity), while a Cambridge score value of ≥0.15 identified a significantly higher proportion of those who developed a first CVD event in the same period (71%, P<0.05).
Table 4 Sensitivity/specificity, positive/negative predictive values and positive likelihood ratio (with 95% CI) of the Framingham risk score and Cambridge diabetes risk score for the prediction of CVD events.
The specificity for the above Framingham score and Cambridge score cut-off points was 74% and 60% respectively. The likelihood ratios for a positive test using the Framingham score cut-off point of 0.20 and the Cambridge score cut-off point of 0.15 were similar (2.49 and 1.77, respectively).
The sensitivity and specificity were also calculated for a stepwise CVD screening programme using a Cambridge score cut-off point of ≥0.15 to identify people at risk (43% of the EPIC-Norfolk population) and then calculating their Framingham score. This approach had lower sensitivity than inviting all individuals for vascular risk assessment, for example, calculating the Framingham score for everyone (54% and 66% respectively) but higher specificity (81% and 74% respectively), with similar false negative rates (6% and 5% respectively). In the sensitivity analysis, excluding family history of diabetes, smoking and/or BMI did not reduce the predictive ability of the Cambridge score.
DISCUSSION
Summary of main findings
To the best of the authors' knowledge, the present study is the first to investigate the performance of a simple diabetes risk score using data that are routinely available in general practice records to assess CVD risk in a UK population. In the EPIC-Norfolk cohort, both the Cambridge risk score and Framingham risk score were moderately effective at identifying those at high risk of developing CVD (discrimination), suggesting that they could be used to target further investigation or preventive interventions to those at highest risk. However, the over-estimation of risk suggests that care is still needed when using risk scores to communicate absolute risk information.
With a moderate discriminative ability and sensitivity/specificity values comparable to the Framingham score, the Cambridge score is an example of how a simple tool using routinely available data might be used as a preliminary step to stratify the population and identify people at high risk of developing CVD. Performance might be improved by development of simple scores specific to CVD.
Strengths and limitations of the study
EPIC-Norfolk is a large, population-based prospective cohort with a long follow-up and low attrition, using standardised ascertainment of CVD events. The predictive performance of different risk assessment tools were evaluated using widely accepted measures20,21 including discrimination, calibration, and sensitivity/specificity, as well as the likelihood ratio, which is robust to changes in the prevalence of disease. However, this study had some limitations.
Misclassification of CVD events might have attenuated the predictive value of the risk scores. Four-fifths of the CVD events were non-fatal and were ascertained via linkage to hospital admission data. Not all non-fatal CVD cases lead to hospital admission, but this method captures the non-fatal events of most clinical importance, and previous validation studies in this cohort indicated high specificity of such case ascertainment.16
Careful consideration should be taken when different definitions of CVD between studies are used. For instance, the Framingham study included heart failure and non-fatal peripheral vascular disease in CVD outcomes,17 but the current study did not.
The study sample was similar to the UK population in terms of anthropometry, blood pressure, and serum lipids; however, given the 33% participation rate in this study, it is possible that attendees might be more likely to seek existing preventive services, compared with non-attendees. Therefore, the overall population incidence of CVD could have been underestimated, which might, in turn, influence estimates of the predictive ability of the risk scores.
Another point of note is that, as EPIC-Norfolk is a predominantly white cohort, the generalisability of these findings may be limited in other ethnic groups.
Comparison with existing literature
These findings suggest that simple risk scores, which do not require laboratory tests, can be incorporated into strategies to predict CVD.
D'Agostino et al recently developed a simple CVD risk assessment model using data routinely obtained in primary care.17 They incorporated BMI instead of total cholesterol and HDL-cholesterol in the original Framingham equations22 and found that the new risk equations had good discriminatory ability with an aROC of 0.75 in men and 0.79 in women. This study showed that a simple diabetes risk score also predicted cardiovascular events reasonably well (aROC 0.69 for men and 0.72 for women).
In agreement with the current results, the Finnish Diabetes Risk Score has also been shown to be a good predictor of coronary heart disease, stroke, and total mortality in a Finnish population.23 However, unlike the Cambridge score, it necessitates distribution and completion of a questionnaire.
A threshold pattern in the ROC curve was not found for the Cambridge score, suggesting no single cut-off point for the prediction of CVD. However, if a cut-off point of 0.15 was chosen, about 71% of individuals who would develop a CVD event within 10 years would be correctly identified (sensitivity). This is more sensitive for the prediction of CVD than using a risk threshold of 20% for the Framingham score in the same population. Only 5% of individuals with a Cambridge score value of <0.15 would develop a CVD event over 10 years (false negative). Moreover, if this cut-off point for CVD risk pre-stratification was used, only two-fifths of the total population would be required to attend their general practice for more time- and resource-consuming screening tests. This is supported by a modelling study by Marshall and Rouse,24 which suggests that strategies preselecting patients for risk assessment may reduce staff time and prevent more new cases within available resources, compared with inviting all individuals. However, the authors used a hypothetical population, assuming default blood pressure and blood-cholesterol values for each individual, and used the Framingham score for pre-stratification.
Implications for future research and clinical practice
This study has shown that the Cambridge score predicts CVD events reasonably well in the EPIC-Norfolk cohort. Despite having slightly reduced discriminatory ability compared with the Framingham score, there is no need for an individual to attend their general practice for blood testing for calculation of the Cambridge score.
It is reassuring that the exclusion of family history of diabetes, smoking, and BMI did not adversely affect the predictive ability of the Cambridge score, suggesting that it can be used in health systems where data on these variables are not routinely collected. The Cambridge score, therefore, provides an example of a simple risk score using routine data that could be used as a preliminary test in a stepwise population screening procedure for CVD risk in primary care. It may be more practical and cost-effective to pre-select a proportion of the population in which more invasive and expensive screening is feasible, rather than inviting every middle-aged adult without known CVD or related conditions, as proposed in the UK and other countries. Such an approach may also reduce potential adverse effects of screening, such as anxiety and false reassurance.8 However, further evaluation to assess the costs and benefits of alternative screening procedures is needed, as well as the development or refinement of simple scores more specific to CVD risk and the mixed ethnic UK population.25
In summary, a simple risk score using data that are routinely available performed reasonably well at predicting CVD risk in the EPIC-Norfolk cohort. Such a score might, therefore, be used as a pre-stratification screening tool in the UK population.
Notes
Funding body
Funding support was provided by the Medical Research Council, Cancer Research UK, British Heart Foundation, European Union (Europe Against Cancer Programme), Stroke Association, Wellcome Trust, Research into Ageing, and the Academy of Medical Science. Dr Chamnan is supported by a Royal Thai Government scholarship. The sponsors did not participate in: the design or conduct of this study; the collection, management, analysis, or interpretation of data; the writing of the manuscript; the preparation or review of this manuscript; the decision to submit it for publication; or its approval for publication.
Ethical approval
The study was approved by the Norwich District Health Authority Ethics Committee. All participants gave signed informed consent.
Competing interests
The authors have stated that there are none.
