Screening for left ventricular systolic dysfunction using GP-reported ECGs

Barclay M Goudie; Rob I Jarvis; Peter T Donnan; Frank M Sullivan; Stuart D Pringle; Sanjay Jeyaseelan; Allan D Struthers

Abstract

Background Diagnostic echocardiography has poor access for patients with suspected heart failure. Pre-echocardiography screening with electrocardiograms (ECGs) is recommended as a means of targeting this scarce resource. There are data to support this policy when ECGs are interpreted by cardiologists but not by GPs.

Aim To assess the value of GP-reported ECGs as a pre-echocardiography screening test for left ventricular systolic dysfunction (LVSD).

Design of study Cross-sectional study of GPs' ECG reporting skills.

Setting General practice, NHS in Scotland.

Method A randomly selected, stratified sample of 123 Scottish GPs reviewed 180 ECGs (100 abnormal, 50 normal and 30 duplicate) from 150 patients with suspected heart failure. Forty-one patients had LVSD on echocardiography. GPs were required to categorise ECGs as normal or abnormal.

Results Mean sensitivity was 0.94 (95% CI = 0.92 to 0.95). Mean specificity 0.58 (95% CI = 0.56 to 0.60). Mean positive predictive value (PPV) was 0.47 (95% CI = 0.46 to 0.48). Mean negative predictive value (NPV) was 0.96 (95% CI = 0.95 to 0.97). Mean likelihood ratio was 2.39 (95% CI = 2.28 to 2.50). Seventy of 123 (57%) GPs achieved sensitivity of 0.9 and specificity of 0.5 for the detection of LVSD.

Conclusion Most Scottish GPs have the skills to perform pre-echocardiography screening ECGs in patients with suspected LVSD. However, differences in ECG reporting performance between individual GPs will result in widely varying referral rates for echocardiography and differences in the detection rate of LVSD. The implications of these findings need to be considered when heart failure diagnostic services are being developed.

INTRODUCTION

Heart failure, caused by left ventricular systolic dysfunction (LVSD), is a major health problem with a prevalence of between 0.4 and 3.2%.1 Accurate diagnosis using echocardiography is a prerequisite to safe cost-effective treatment.2^,3 However, the number of patients consulting GPs with symptoms suggestive of LVSD greatly exceeds the capacity of existing echocardiography services. A screening test is required to identify the subgroup of patients in whom echocardiography is cost-effective.

In 2003, the National Institute for Health and Clinical Excellence (NICE) recommended that the electrocardiogram (ECG) and B-type natriuretic peptide tests should be used as pre-echocardiography screening tests for LVSD.3 NICE guidance did not indicate whether B-type natriuretic peptide testing alone, ECG alone, or a combination of both should be performed. A recent systematic review and meta-analysis concluded that the accuracy of both tests is comparable and that either can be used; there is no evidence to justify the use of both together.1

Patients with an abnormal recording should be referred for echocardiography. In practice the ECG would be interpreted by the GP during the initial assessment of patients with symptoms suggestive of heart failure. However, there is currently no evidence that the pre-echocardiography ECG is a reliable screening test for LVSD when reported by GPs.

How this fits in

Poor access to diagnostic echocardiography is the major impediment to cost-effective treatment of patients with left ventricular systolic dysfunction (LVSD). Pre-echocardiography electrocardiogram (ECG) screening is useful in targeting this scarce resource when performed by cardiologists. Most GPs have the skills needed to perform pre-echocardiography screening using ECGs. However, differences in ECG-reporting performance between individual GPs would result in varying referral rates for echocardiography and differences in the detection rate of LVSD. These findings have implications for the development and implementation of diagnostic heart failure services.

The aim of this study was to assess the value of GP-interpreted ECGs as a pre-echocardiography screening test in a large randomly selected group of GPs from practices throughout Scotland.

METHOD

Population studied

A random sample of 500 GPs was identified from the current list of Scottish GP principals (3784 GPs in total) which is held by the Information and Statistics Division (ISD Scotland). Members of the project team were removed from the list. Approval to contact individual GPs was obtained from health boards.

The study was performed between November 2004 and July 2005. Each GP was contacted by letter, and asked to complete a previously piloted questionnaire to obtain information about cardiology experience, whether they had obtained the MRCP number of ECGs read per month, and confidence in reporting ECGs. This information was used to stratify the study population into three subgroups with low (group 1), medium (group 2) and high (group 3) levels of ECG interpreting experience and confidence. Of the responders, 32% (132/418) had low scores, 46% (193/418) medium, and 22% (93/418) had high scores.

Researchers aimed to recruit 140 GPs with a representative range of confidence and experience in ECG interpretation. Invitations were issued in batches until 44 GPs with low scores, 65 with medium, and 31 with high experience and confidence scores had agreed to participate or until all GPs in the subgroup had been invited.

Study procedure

Each GP who agreed to participate was asked to review 180 ECGs from patients with suspected heart failure, 30 of which were duplicate ECGs. These patients had been referred to the community-based open access echocardiogram service from September 2002 to August 2003.4 Fifty of the ECGs had previously been reported by a cardiologist as normal, 50 had a minor abnormality, and 50 a major abnormality. Of the 150 ECGs, 41 corresponded to LVSD on echocardiography. Forty LVSD cases showed major ECG abnormalities and one had a minor abnormality. Major abnormalities were defined as previous myocardial infarction, left ventricular hypertrophy, left bundle branch block, atrial fibrillation, and left axis deviation.5 Minor abnormalities included sinus brady/tachycardia, first-degree heart block, right bundle branch block, right axis deviation, poor R-wave progression, atrial ectopics, ventricular ectopics, and non-specific ST-T wave changes. Thirty duplicate ECGs were given to GPs to assess consistency of reporting (10 normal, 10 with a minor abnormality, and 10 with a major abnormality). The order of the ECGs was mixed before being given to GPs.

GPs were informed that all ECGs were taken from patients with symptoms suggestive of LVSD. They were asked to indicate on a multiple-choice response sheet whether, in their judgment, the ECG was normal or abnormal, without consulting colleagues or textbooks. They were not asked to interpret the ECGs to achieve a particular level of sensitivity or specificity for the detection of LVSD. A fee of £120 was paid to each GP who returned a completed multiple choice response sheet. Data were double entered into an Excel spread sheet by two data entry staff working independently. Inconsistencies in data entry were corrected by referring to the original data. Procedures and materials were piloted in a small study of six GPs who were excluded from the final project.6

Statistical methods

Performance measures in detecting LVSD were sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and likelihood ratio. Results were presented as means and 95% confidence intervals (95% CI). κ statistics were calculated for consistency in reporting the same ECGs on a second occasion. One-way analysis of variance (ANOVA) was used to compare mean performance levels between groups of GPs defined by experience and confidence. Statistical analysis was performed using SPSS (version 10.1).

RESULTS

Response rate

There was an 84% (418/500) response rate to the initial confidence and experience questionnaire. Of those who were invited to interpret ECGs, 123 GPs interpreted ECGs and returned completed response sheets (group 1, low: n = 35; group 2, medium: n = 59; and group 3, high: n = 29; Figure 1). These proportions reflect the spectrum of experience and confidence in the study GP population.

Figure 1

Recruitment of GPs to interpret ECGs.

Sensitivity and specificity for LVSD

Mean sensitivity of GP-reported ECGs as a pre-echocardiography screening test for LVSD was 0.94 (95% CI = 0.92 to 0.95). ECG screening had a lower sensitivity when performed by less experienced and less confident GPs: group 1 mean sensitivity 0.90 (95% CI = 0.87 to 0.93); group 2 mean sensitivity 0.95 (95% CI = 0.94 to 0.97); group 3 mean sensitivity 0.95 (95% CI = 0.92 to 0.97) ANOVA F = 6.41, P = 0.002 (Figure 2).

Figure 2

Mean ECG sensitivity and specificity (%) for left ventricular systolic dysfunction (LVSD) by GPs' level of experience and confidence in detection.

Mean specificity of GP-reported ECGs as a pre-echocardiography screening test for LVSD was 0.58 (95% CI = 0.56 to 0.60). There was no significant difference in specificity for LVSD between the groups with different levels of experience/confidence, ANOVA F = 1.62, P = 0.20 (Figure 2).

Predictive values for LVSD

The prevalence of LVSD was 27.3% (41/150) in the population of patients whose ECGs were interpreted in this study. Mean PPV of GP-reported ECGs as a pre-echocardiography screening test for LVSD was 0.47 (95% CI = 0.46 to 0.48). Mean NPV of GP-reported ECGs as a pre-echocardiography screening test for LVSD was 0.96 (95% CI = 0.95 to 0.97). In a population with LVSD prevalence of 10%, the PPV would be 0.21 (95% CI = 0.20 to 0.22) and the NPV would be 0.99 (95% CI = 0.987 to 0.991). With a LVSD prevalence of 20%PPV would be 0.37 (95%CI = 0.36 to 0.38) and NPV would be 0.92 (95% CI = 0.90 to 0.93). The mean likelihood ratio (sensitivity/[1-specificity]) was 2.39 (95% CI = 2.28 to 2.50).

Mean κ statistic for intra-rater reliability was 0.69 (95% CI = 0.65 to 0.72) demonstrating good consistency in ECG reporting by individual GPs. There was no significant difference in κ statistic between experience/confidence groups (ANOVA F = 0.43, P = 0.65).

Assessment of current performance

The outcome of screening for LVSD by individual GPs is shown in Figure 3. Differences in ECG reporting performance between GPs would result in variation in referral rates for echocardiography and differences in the detection rate of LVSD. The current data indicate that 70/123 (57%) GPs achieved a sensitivity of 0.9 and specificity of 0.5 for the detection of LVSD.

Figure 3

Screening performance of individual GPs. True positive rate and false positive rate for LVSD. Standard set at sensitivity = 90% and specificity = 50%. Number of GPs noted in each quadrant. Total n = 123.

Experience of and confidence with ECG interpreting, as assessed by the initial questionnaire, was a poor predictor of screening performance in individual GPs. Members of all three groups failed to achieve a sensitivity for LVSD > or ≥0.9 or specificity for LVSD > or ≥0.5. The proportions not achieving this standard were as follows: group 1 = 21/35 (60%; 95% CI = 43 to 77%), group 2 = 20/59 (34%; 95%CI = 21 to 46%), group 3 = 12/29 (41%; 95% CI = 22 to 60%).

DISCUSSION

Summary of main findings

Pre-echocardiography ECG screening for LVSD by a representative sample of Scottish GPs has a mean sensitivity of 0.94 (95%CI = 0.92 to 0.95) and a mean specificity of 0.58 (95% CI = 0.56 to 0.60). Quality of screening, which is dependent on ECG reporting skill, varies between individual GPs. Fifty-seven per cent of GPs achieve a sensitivity of 90% and a specificity of 50% for the detection of LVSD.

Strengths and limitations of the study

Participants are a representative sample of GPs. They were randomly selected from the current list of Scottish GP principals and stratified to ensure that they reflect the range of ECG-reporting experience and confidence in the GP population. ECGs were all obtained from patients consulting their GPs with symptoms suggestive of heart failure.

It is necessary to consider the effect of case mix when comparing the current results with those of other pre-echocardiography screening studies. This study included 41 ECGs from patients with LVSD among the 150 ECGs in the study. Of these, 40 had a major ECG abnormality and one (2.4%) had a minor abnormality. This contrasts with a study where ECGs showed minor abnormalities or were normal in 6% of LVSD cases,5 and others where 10–12% of cardiologist-reported ECGs were normal.7^,8 The absence of LVSD cases with normal ECGs will result in an overestimate of the sensitivity of GP screening in the current study. The inclusion of only one case with a minor ECG abnormality may also result in a small overestimate of the sensitivity of GP screening. GPs may have more difficulty distinguishing minor abnormalities from normal, than major abnormalities from normal. They are more likely to miss a case of LVSD with a minor ECG abnormality. The possibility that some GPs may have consulted textbooks or colleagues while interpreting ECGs cannot be excluded.

Comparison with existing literature

The potential of the ECG as a pre-echocardiography screening test for LVSD has been demonstrated in studies where ECGs were interpreted by hospital specialists.5^,7^,9^,10 Prior to this study, data were limited on pre-echocardiography screening for LVSD with GP-reported ECGs,6^,11^,12 and it was not possible to make informed or evidence-based decisions about its place in a national strategy for the diagnosis of heart failure. In the present study, where pre-echocardiography ECG screening for LVSD was performed by a representative sample of Scottish GPs, mean sensitivity and specificity are comparable to those published for cardiologist-reported ECGs (pooled sensitivity 0.90 [95% CI = 0.88 to 0.92] and pooled specificity 0.58 [95% CI = 0.56 to 0.60]).13

Implications for clinical practice

Results from the current study indicate that overall screening performance is satisfactory. However, differences in the quality of ECG reporting between individual GPs will result in variation in referral rates for echocardiography and in detection rates of LVSD. Managers with a responsibility for implementing the NICE guideline3 need to be aware of these results. They may consider it appropriate to allow referral of patients in whom there was a high clinical suspicion of LVSD, even when the GP considers the ECG to be normal. Training that uses appropriate educational packages could increase the effectiveness of pre-echocardiography screening using GP-reported ECGs, although there is no evidence to support this at present. Alternatively, improved access to B-type natriuretic peptide testing, which is being evaluated in primary care patients14 and has been recommended for use in NHS in Scotland,13 could be a preferred means of achieving more uniform patterns of referral for echocardiography.

Acknowledgments

We would like to thank Margaret Feeney and Marie Pitkethly of EastRen for administrative and secretarial support, and to our GP colleagues for participating in the project.

Notes

Funding body

This project was funded by NHS in Scotland Chief Scientist Office. Sanjay Jeyaseelan was funded by the British Heart Foundation

Ethics committee

This study received MREC approval as a ‘no local investigator’ study from Northwest Multi-centre Research Ethics Committee at their meeting of 8 June 2004 (04/MRE08/2)

Competing interests

The authors have stated that there are none

Received May 2, 2006.
Revision received July 6, 2006.
Accepted October 25, 2006.

REFERENCES

↵
1. Davenport C,
2. Cheng EYL,
3. Kwok YTT,
4. et al.
(2006) Assessing the diagnostic test accuracy of natriuretic peptides and ECG in the diagnosis of left ventricular systolic dysfunction: a systematic review and meta-analysis. Br J Gen Pract 56(522):48–56.
OpenUrl Abstract/FREE Full Text
↵
1. Scottish Intercollegiate Guidelines Network
(1999) Guidelines for diagnosis and treatment of heart failure due to left ventricular systolic dysfunction (SIGN, Edinburgh).
↵
1. National Institute for Clinical Excellence
(2003) Chronic heart failure: national clinical guideline for diagnosis and management in primary and secondary care (NICE, London).
↵
1. Jeyaseelan S,
2. Goudie BM,
3. Pringle SD,
4. et al.
(2005) Agreement between community echocardiography and hospital echocardiography in patients suspected of having left ventricular systolic dysfunction. Postgrad Med J 81(962):777–779.
OpenUrl PubMed
↵
1. Davie AP,
2. Francis CM,
3. Love MP,
4. et al.
(1996) Value of the electrocardiogram in identifying heart failure due to left ventricular systolic dysfunction. BMJ 312(7025):222.
OpenUrl FREE Full Text
↵
1. Jeyaseelan S,
2. Struthers AD,
3. Goudie BM,
4. et al.
(2006) The accuracy of ECG screening by general practitioners and by ECG machine interpretation in selecting suspected heart failure patients for echocardiography. Br J Cardiol 13(3):216–218.
OpenUrl
↵
1. Lindsay MM,
2. Goodfield NER,
3. Hogg KJ,
4. et al.
(2000) Optimising direct access ECHO referral in suspected heart failure. Scott Med J 45(2):43–44.
OpenUrl CrossRef PubMed
↵
1. Ng LL,
2. Loke I,
3. Davies JE,
4. et al.
(2003) Identification of previously undiagnosed left ventricular systolic dysfunction: community screening using natriuretic peptides and electrocardiography. Eur J Heart Fail 5(6):775–782.
OpenUrl CrossRef PubMed
↵
1. Gillespie ND,
2. McNeil G,
3. Pringle T,
4. et al.
(1997) Cross-sectional study of contribution of clinical assessment and simple cardiac investigations to diagnosis of left ventricular systolic dysfunction in patients admitted with acute dyspnoea. BMJ 314(7085):936–940.
OpenUrl Abstract/FREE Full Text
↵
1. Hutcheon SD,
2. Gillespie ND,
3. Struthers AD,
4. et al.
(2002) B-type natriuretic peptide in the diagnosis of cardiac disease in elderly day hospital patients. Age Ageing 31(4):295–301.
OpenUrl CrossRef PubMed
↵
1. Sandler DA,
2. Steeds RP,
3. Hadfield JW
(2000) Open access to echocardiography for general practitioners in North Derbyshire. Br J Cardiol 7:94–100.
OpenUrl
↵
1. Houghton AR,
2. Sparrow NJ,
3. Toms E,
4. et al.
(1997) Should general practitioners use the electrocardiogram to select patients with suspected heart failure for echocardiography? Int J Cardiol 62(1):31–36.
OpenUrl CrossRef PubMed
↵
1. Craig J,
2. Bradbury I,
3. Cummins E,
4. et al.
(2005) The use of B-type natriuretic peptides (BNP and NT-proBNP) in the investigation of patients with suspected heart failure (NHS Quality Improvement Scotland, Edinburgh) Health Technology Assessment Report 6 – May 2005.
↵
1. Fuat A,
2. Murphy JJ,
3. Hungin APS,
4. et al.
(2006) The diagnostic accuracy and utility of a B-type natriuretic peptide test in a community population of patients with suspected heart failure. Br J Gen Pract 56(526):327–333.
OpenUrl Abstract/FREE Full Text

In this issue

Download PDF

Download PowerPoint

Email Article

Citation Tools

Keywords

More in this TOC Section

Show more Original Papers

Cited By...

Intended for Healthcare Professionals

[1] ↵
Davenport C,
Cheng EYL,
Kwok YTT,
et al.
(2006) Assessing the diagnostic test accuracy of natriuretic peptides and ECG in the diagnosis of left ventricular systolic dysfunction: a systematic review and meta-analysis. Br J Gen Pract 56(522):48–56.
OpenUrl Abstract/FREE Full Text

[2] Davenport C,

[3] Cheng EYL,

[4] Kwok YTT,

[5] et al.

[6] ↵
Scottish Intercollegiate Guidelines Network
(1999) Guidelines for diagnosis and treatment of heart failure due to left ventricular systolic dysfunction (SIGN, Edinburgh).

[7] Scottish Intercollegiate Guidelines Network

[8] ↵
National Institute for Clinical Excellence
(2003) Chronic heart failure: national clinical guideline for diagnosis and management in primary and secondary care (NICE, London).

[9] National Institute for Clinical Excellence

[10] ↵
Jeyaseelan S,
Goudie BM,
Pringle SD,
et al.
(2005) Agreement between community echocardiography and hospital echocardiography in patients suspected of having left ventricular systolic dysfunction. Postgrad Med J 81(962):777–779.
OpenUrl PubMed

[11] Jeyaseelan S,

[12] Goudie BM,

[13] Pringle SD,

[14] et al.

[15] ↵
Davie AP,
Francis CM,
Love MP,
et al.
(1996) Value of the electrocardiogram in identifying heart failure due to left ventricular systolic dysfunction. BMJ 312(7025):222.
OpenUrl FREE Full Text

[16] Davie AP,

[17] Francis CM,

[18] Love MP,

[19] et al.

[20] ↵
Jeyaseelan S,
Struthers AD,
Goudie BM,
et al.
(2006) The accuracy of ECG screening by general practitioners and by ECG machine interpretation in selecting suspected heart failure patients for echocardiography. Br J Cardiol 13(3):216–218.
OpenUrl

[21] Jeyaseelan S,

[22] Struthers AD,

[23] Goudie BM,

[24] et al.

[25] ↵
Lindsay MM,
Goodfield NER,
Hogg KJ,
et al.
(2000) Optimising direct access ECHO referral in suspected heart failure. Scott Med J 45(2):43–44.
OpenUrl CrossRef PubMed

[26] Lindsay MM,

[27] Goodfield NER,

[28] Hogg KJ,

[29] et al.

[30] ↵
Ng LL,
Loke I,
Davies JE,
et al.
(2003) Identification of previously undiagnosed left ventricular systolic dysfunction: community screening using natriuretic peptides and electrocardiography. Eur J Heart Fail 5(6):775–782.
OpenUrl CrossRef PubMed

[31] Ng LL,

[32] Loke I,

[33] Davies JE,

[34] et al.

[35] ↵
Gillespie ND,
McNeil G,
Pringle T,
et al.
(1997) Cross-sectional study of contribution of clinical assessment and simple cardiac investigations to diagnosis of left ventricular systolic dysfunction in patients admitted with acute dyspnoea. BMJ 314(7085):936–940.
OpenUrl Abstract/FREE Full Text

[36] Gillespie ND,

[37] McNeil G,

[38] Pringle T,

[39] et al.

[40] ↵
Hutcheon SD,
Gillespie ND,
Struthers AD,
et al.
(2002) B-type natriuretic peptide in the diagnosis of cardiac disease in elderly day hospital patients. Age Ageing 31(4):295–301.
OpenUrl CrossRef PubMed

[41] Hutcheon SD,

[42] Gillespie ND,

[43] Struthers AD,

[44] et al.

[45] ↵
Sandler DA,
Steeds RP,
Hadfield JW
(2000) Open access to echocardiography for general practitioners in North Derbyshire. Br J Cardiol 7:94–100.
OpenUrl

[46] Sandler DA,

[47] Steeds RP,

[48] Hadfield JW

[49] ↵
Houghton AR,
Sparrow NJ,
Toms E,
et al.
(1997) Should general practitioners use the electrocardiogram to select patients with suspected heart failure for echocardiography? Int J Cardiol 62(1):31–36.
OpenUrl CrossRef PubMed

[50] Houghton AR,

[51] Sparrow NJ,

[52] Toms E,

[53] et al.

[54] ↵
Craig J,
Bradbury I,
Cummins E,
et al.
(2005) The use of B-type natriuretic peptides (BNP and NT-proBNP) in the investigation of patients with suspected heart failure (NHS Quality Improvement Scotland, Edinburgh) Health Technology Assessment Report 6 – May 2005.

[55] Craig J,

[56] Bradbury I,

[57] Cummins E,

[58] et al.

[59] ↵
Fuat A,
Murphy JJ,
Hungin APS,
et al.
(2006) The diagnostic accuracy and utility of a B-type natriuretic peptide test in a community population of patients with suspected heart failure. Br J Gen Pract 56(526):327–333.
OpenUrl Abstract/FREE Full Text

[60] Fuat A,

[61] Murphy JJ,

[62] Hungin APS,

[63] et al.

Main menu

User menu

Search