Optimising prediction of mortality, stroke, and major bleeding for patients with atrial fibrillation: validation of the GARFIELD-AF tool in UK primary care electronic records

Background The GARFIELD-AF tool is a novel risk tool that simultaneously assesses the risk of all-cause mortality, stroke or systemic embolism, and major bleeding in patients with atrial fibrillation (AF). Aim To validate the GARFIELD-AF tool using UK primary care electronic records. Design and setting A retrospective cohort study using the Clinical Practice Research Datalink (CPRD) linked with Hospital Episode Statistics data and Office for National Statistics mortality data. Method Discrimination was evaluated using the area under the curve (AUC) and calibration was evaluated using calibration-in-the-large regression and calibration plots. Results A total of 486 818 patients aged ≥18 years with incident diagnosis of non-valvular AF between 2 January 1998 and 31 July 2020 were included; 50.6% (n = 246 425/486 818) received anticoagulation at diagnosis The GARFIELD- AF models outperformed the CHA2DS2VASc and HAS-BLED scores in discrimination ability of death, stroke, and major bleeding at all the time points. The AUC for events at 1 year for the 2017 models were: death 0.747 (95% confidence interval [CI] = 0.744 to 0.751) versus 0.635 (95% CI = 0.631 to 0.639) for CHA2DS2VASc; stroke 0.666 (95% CI = 0.663 to 0.669) versus 0.625 (95% CI = 0.622 to 0.628) for CHA2DS2VASc; and major bleeding 0.602 (95% CI = 0.598 to 0.606) versus 0.558 (95% CI = 0.554 to 0.562) for HAS- BLED. Calibration between predicted and Kaplan– Meier observed events was inadequate with the GARFIELD-AF models. Conclusion The GARFIELD-AF models were superior to the CHA2DS2VASc score for discriminating stroke and death and superior to the HAS-BLED score for discriminating major bleeding. The models consistently underpredicted the level of risk, suggesting that a recalibration is needed to optimise its use in the UK population.


INTRODUCTION
Oral anticoagulation (OAC) substantially reduces the risk of AF-related stroke. 1 However, OAC increases the risk of bleeding, and AF management guidelines recommend the use of risk stratification tools to guide decisions on anticoagulation. 2,3he European Society of Cardiology and the National Institute for Health and Care Excellence (NICE) AF guidelines recommend the CHA 2 DS 2 VASc score for assessing stroke risk. 3Until recently, both guidelines recommended the HAS-BLED score for accessing bleeding risk; however, since 2021 NICE has recommended the ORBIT-AF risk score. 3he recommended scores are widely used in clinical practice, nevertheless up to 15% of patients with AF at risk of stroke in England do not receive guideline- recommended therapy. 4The GARFIELD-AF risk tool is a novel risk tool that simultaneously assesses a patient's risk of mortality, stroke or systemic embolism, and risk of major bleeding. 5,6he GARFIELD-AF tool was developed based on 39 898 patients enrolled on the GARFIELD- AF registry in 2017, 5 and a new version published in 2021 predicted events up to 2 years from diagnosis. 6Initial evaluations indicate that both versions are superior to CHA 2 DS 2 VASc in predicting ischaemic stroke/systemic embolism and superior to HAS-BLED in predicting bleeding risk. 5,6ARFIELD-AF is an international prospective observational study of patients aged ≥18 years with newly diagnosed AF and ≥1 investigator-determined risk factor for stroke. 7,8There were a total of 52 080 participants enrolled in 35 countries who were followed for a minimum of 2 years; 3574 of the GARFIELD-AF cohort were recruited in the UK. 9 The GARFIELD-AF tool can potentially be embedded into primary care electronic systems to aid decision making regarding anticoagulation so that patients who require anticoagulation receive it and those that do not need it do not receive it.
The performance of the prediction model tends to vary across settings and populations, and external validation is required to fully appreciate the generalisability of a prediction model. 10,11he purpose of this study was to validate the GARFIELD-AF tool in patients with AF in an NHS primary care electronic health records database, and compare the performance of the GARFIELD-AF tool with the CHA 2 DS 2 VASc and HAS-BLED scores.

Source of data
The primary data source was the Clinical Practice Research Datalink (CPRD), an electronic primary care database comprising anonymised patient medical records from GPs, with coverage of over 19 million patients from 738 practices in the UK. 12 Data were extracted by CPRD and linked with Hospital Episode Statistics (HES) data, which provides information on all hospital admissions, and mortality data from the Office for National Statistics (ONS).

Study population
The study population was defined as adults aged ≥18 years with incident diagnosis of non-valvular AF between 2 January 1998 and 31 July 2020, and eligible for linkage with HES and ONS data.

Follow up
Start of follow up was defined as the recorded date the patient was diagnosed with non-valvular AF.End of follow up was defined as death as recorded by ONS, end of practice registration, or last collection date, whichever occurred first.

Covariates
The covariates for the GARFIELD-AF models are: age, sex, pulse, systolic blood pressure and diastolic blood pressure, weight, height, ethnicity, current smoking, and paroxysmal AF; history of vascular disease, diabetes, cirrhosis, peripheral vascular disease, stroke, bleeding, heart failure, chronic kidney disease, sleep apnoea, dementia, and/or carotid occlusive disease; and anticoagulant use and antiplatelet use.The covariates and coefficients for the 2017 and 2021 models are detailed in Supplementary Tables S1 and S2.
The main difference between the 2017 and the 2021 models is that the 2021 models have a wider range of variables.For example, the 2017 GARFIELD-AF model for stroke includes the variables age, history of stroke, bleeding, heart failure, chronic kidney disease, region, ethnicity, and anticoagulant use.The baseline variables for the GARFIELD-AF models and CHA 2 DS 2 VASc and HAS-BLED scores were defined from CPRD data using Medical Code IDs.Details are provided in Supplementary Box S1.

Definition of endpoints
The study endpoints were all-cause mortality; ischaemic stroke/systemic embolism, defined as the combined endpoint of any ischaemic stroke, transient ischaemic attack, or systemic embolism; and major bleeding (including haemorrhagic stroke), defined as bleeding requiring admission to hospital.The first occurrence of an ischaemic stroke/systemic embolism after AF diagnosis was the endpoint for ischaemic stroke/systemic embolism, and the first occurrence of major bleeding after AF diagnosis was the endpoint for major bleeding.

Outcome variables
Outcome variables were defined from both Medical Code IDs and International Classification of Diseases 10th Revision codes for HES and ONS mortality data, as detailed in Supplementary Box S2.

How this fits in
Anticoagulation reduces the risk of atrial fibrillation (AF)-related stroke at the cost of an increased risk of bleeding.The CHA 2 DS 2 VASc score is used to assess stroke risk in patients with AF, whereas the HAS-BLED or ORBIT-AF scores are used to assess bleeding risk.A novel tool, GARFIELD-AF simultaneously predicts the risk of stroke death and bleeding in patients with AF; however, its performance has not been tested in the UK population.The GARFIELD-AF models had better discriminatory ability than the CHA 2 DS 2 VASc and HAS-BLED scores in the UK population; however, it underestimated the level of risk.

Statistical analysis
The GARFIELD-AF models were applied to the CPRD dataset to obtain the predicted risks for each outcome.The performance of the tool was measured in terms of calibration using calibration-in-the-large regression and calibration plots, and in terms of discrimination using the area under the receiver operating characteristic curve (AUC), also referred to as the C-statistic.The performance of the models was compared with the CHA 2 DS 2 VASc and HAS-BLED scores by comparing the AUC of each model.The CHA 2 DS 2 VASc score, in addition to predicting the risk of stroke in patients with AF, has been shown to predict mortality in patients with several diseases, regardless of the presence of AF. 13 The performance of the CHA 2 DS 2 VASc score for predicting stroke and death was compared with the GARFELD-AF models for stroke and death, and the performance of the HAS-BLED score for predicting bleeding was compared with the GARFIELD-AF bleeding models.The treatment effect was estimated by running separate Cox regression models for each outcome (death, stroke, and bleeding) and adjusting each model for all the variables that contribute to the GARFIELD-AF 2021 score for that outcome.
Each variable was assessed for the degree of missingness.The assessment for discrimination and calibration was performed on the whole dataset and repeated in patients without missing data in any score.Subgroup analysis was conducted according to risk stratification of stroke (high, moderate, and low according to CHA 2 DS 2 VASc) and bleeding (HAS-BLED <2 or >2), and for individuals receiving anticoagulation or no anticoagulation at baseline.

RESULTS
A total of 708 474 patients had an incident record of AF in CPRD Aurum.Of these, 486 818 met the inclusion criteria for the study (Figure 1).The median follow up was 3.975 years (interquartile range 1.6-7.7;minimum 0 to maximum 22.6).

Baseline characteristic of participants
The baseline characteristics for the CPRD validation cohort, the UK GARFIELD-AF subcohort, and the global GARFIELD-AF cohort are presented in Table 1

External validation for the GARFIELD-AF models
Table 2 shows the full data for the 2017 1-year mortality, stroke, and bleeding models and the 2021 models (each model with 1-month, 1-year, and 2-year follow up).

Discrimination
The AUCs in Table 2

Calibration
For the three outcomes, both the 2017 and 2021 GARFIELD-AF models consistently predicted less average risk than the observed risks in the population estimated using the Kaplan-Meier method (Table 2 and Figure 2).The 2017 tool performed slightly better than the 2021 tool at 1-year follow up for the three outcomes.The calibration plots show that the differences between the GARFIELD-AF's predicted risks and the Kaplan-Meier estimated risks grow in the larger quintiles (Figure 3).

Comparison with GARFIELD-AF models and CHA 2 DS 2 VASc and HAS-BLED scores
The GARFIELD-AF models consistently outperformed the CHA 2 DS 2 VASc and HAS-BLED scores (Table 3).The AUC for the 2017 models at

Subgroup analyses
Patients not taking OAC showed a higher average risk of events in almost every version of the model than those patients taking OAC.The AUC was always larger in patients not taking OAC.This is compatible with OAC lowering the risks of patients and making it more difficult to tell who is going to have an event (lower AUC) (see Supplementary Table S3).
After adjusting for the GARFIELD-AF 2021 risk factors in Cox regression models, anticoagulation had a protective effect from death with an adjusted hazard ratio (aHR) 0.58 (95% CI = 0.50 to 0.68), a protective effect for stroke with an  When stratified according to risk levels, the GARFIELD-AF models performed better in patients at high risk compared with moderate risk for stroke according to CHA 2 DS 2 VASc (see Supplementary Table S3).The AUCs for 2017 1-year risk for the stroke model were: high risk 0.652 (95% CI = 0.649 to 0.656), moderate risk 0.559 (95% CI = 0.545 to 0.572), and low risk 0.526 (95% CI = 0.508 to 0.543).

a
The risk factor labile international normalised ratio is not included in the HAS-BLED score.As a result, the maximum HAS-BLED score at baseline is 8 points (not 9).bDenominators of the medical history risk factors vary depending on how many individuals had the information available.The percentages are calculated based on the number of people with information in each risk factor (not shown).AF = atrial fibrillation.AP = antiplatelet.BMI = body mass index.CPRD = Clinical Practice Research Datalink.NOAC = non-vitamin K antagonist oral anticoagulant.NSAID = non-steroidal anti-inflammatory drug.OAC = oral anticoagulation.SD = standard deviation.TIA = transient ischaemic attack.VKA = vitamin K antagonist.British Journal of General Practice, November 2023 e819

Figure 2 .
Figure 2. Predicted versus Kaplan-Meier risk of the GARFIELD-AF models.f = full.m = month.

Figure 3 .
Figure 3. Calibration plots for death, stroke, and bleeding outcomes by quantiles of predicted risk.KM = Kaplan-Meier.

Table 2 . Predicted and Kaplan-Meier estimated risks for the GARFIELD-AF models
AUC = area under the curve (C-statistic).Full = full version of the 2017 death model.KM = estimated risk using Kaplan-Meier method.N0 = number of patients without the outcome atend of follow up.P0 = average predicted risk in patients in N0.N1 = number of patients with positive outcome at end of follow up.P1 = average predicted risk in patients in N1.