Research

Chest pain for coronary heart disease in general practice: clinical judgement and a clinical decision rule

Jörg Haasenritter, Norbert Donner-Banzhoff and Stefan Bösner
British Journal of General Practice 2015; 65 (640): e748-e753. DOI: https://doi.org/10.3399/bjgp15X687385

Abstract

Background The Marburg Heart Score (MHS) is a simple, valid, and robust clinical decision rule assisting GPs in ruling out coronary heart disease (CHD) in patients presenting with chest pain.

Aim To investigate whether using the rule adds to the GP’s clinical judgement.

Design and setting A comparative diagnostic accuracy study was conducted using data from 832 consecutive patients with chest pain in general practice.

Method Three diagnostic strategies were defined using the MHS: diagnosis based solely on the MHS; using the MHS as a triage test; and GP’s clinical judgement aided by the MHS. Their accuracy was compared with the GPs’ unaided clinical judgement.

Results Sensitivity and specificity of the GPs’ unaided clinical judgement was 82.9% (95% confidence interval [CI] = 72.4 to 89.9) and 61.0% (95% CI = 56.7 to 65.2), respectively. In comparison, the sensitivity of the MHS was higher (difference 8.5%, 95% CI = −2.4 to 19.6) and the specificity was similar (difference −0.4%, 95% CI = −5.3 to 4.5); the sensitivity of the triage was similar (difference −1.5%, 95% CI = −9.8 to 7.0) and the specificity was higher (difference 11.6%, 95% CI = 7.8 to 15.4); and both the sensitivity and specificity of the aided clinical judgement were higher (difference 8.0%, 95% CI = −6.9 to 23.0 and 5.8%, 95% CI = −1.6 to 13.2, respectively).

Conclusion Using the Marburg Heart Score for initial triage can improve the clinical diagnosis of CHD in general practice.

INTRODUCTION

Chest pain remains a diagnostic challenge in general practice. Although the underlying aetiology in the majority of patients is non-cardiac, for example, musculoskeletal, psychological, or oesophageal, coronary heart disease (CHD) accounts for 8.0–14.6% of chest pain cases in this setting.1–3 GPs must reliably identify serious cardiac disease while also protecting patients from unnecessary investigations and hospital admissions. Based on the results of the medical interview and physical examination, the GP determines the probability of CHD and decides whether additional tests or treatment are needed.

A simple prediction rule assisting GPs to identify patients with a low probability of CHD as the underlying cause of chest pain was developed.4 The Marburg Heart Score (MHS) is based on five findings of the medical history and physical examination; all five items contribute equally to the score (Table 1). The MHS was externally validated in two independent samples.4,5 Among the patients with a negative result (≤2 points), the prevalence of CHD was 2.3%4 and 2.1%.5 However, it remains unclear whether using the rule adds to the GP’s clinical judgement. Furthermore, the placement of the MHS in the sequence of diagnostic reasoning about patients with chest pain is uncertain.

View this table:
Table 1.

Components of the Marburg Heart Score

In this study, the data of the second validation study5 were utilised to investigate whether using the rule adds to the GP’s clinical judgement. In order to reach that aim different diagnostic strategies using the MHS were defined and their accuracy compared with the GPs’ clinical judgement.

METHOD

A comparative diagnostic accuracy study was conducted. Over a period of 12 weeks, 56 GPs located in the state of Hesse, Germany, consecutively recruited patients with chest pain. Patients were eligible if they had pain localised in the anterior chest as primary or secondary complaint and if they were aged ≥35 years. Patients were excluded if chest pain had subsided for more than 1 month, had already been investigated, or was caused by a trauma. Patients with a known history of CHD were not excluded. Data were collected between July 2009 and August 2010. Study audits in each practice ensured consecutive recruitment and complete recording of study data. During the index consultation, study physicians gathered standardised data on the medical history and clinical examination including the variables contributing to the MHS.

Index tests

Of the 56 study physicians, 39 were unaware of the MHS (subsample A). After a medical interview and physical examination, but before conducting further tests, these physicians rated the probability of CHD as the underlying cause using a three-level scale:

  • 0 ‘low probability: no further diagnostic or therapeutic steps regarding CHD are recommended’;

  • 1 ‘intermediate probability: CHD is not the most likely cause of the chest pain, but further diagnostic steps are recommended to rule out CHD’; and

  • 2 ‘high probability: CHD is the most likely cause of the chest pain’.

How this fits in

Previous studies showed that the Marburg Heart Score is a valid, robust, and easy-to-use tool for ruling out CHD in patients presenting with chest pain in general practice. In addition, the current study showed its potential to improve the established diagnostic approach. Considering these points, the authors recommend its use in clinical practice.

The physicians used their discretion when deciding on which clinical findings to base their judgement. This test strategy, which represents the ‘established’ diagnostic test in this situation, was referred to as ‘unaided clinical judgement’.

Three alternative test strategies were defined using the MHS:

  • The established test strategy, the physician’s unaided judgement, was replaced with the MHS. This test strategy is referred to as ‘MHS’.

  • The physician’s clinical judgement and the MHS were combined in a formal and standardised way using the latter as a triage test. Patients with definite MHS results were counted as negative (score ≤2 points) or positive (score ≥4 points). In patients with an intermediate MHS score (3 points), the final test result was determined by the physicians’ unaided clinical judgement. This test strategy is referred to as ‘triage’.

  • In the third alternative a more integrative and less standardised implementation of the MHS was defined in clinical practice. The remaining 17 study physicians (subsample B) participated in the evaluation of a new clinical practice guideline for chest pain by the German College of General Practitioners and Family Physicians (DEGAM). The MHS is part of the guideline. These physicians were instructed to explicitly calculate the MHS score and to consider the results when rating the probability of CHD using the same three-level scale. Likewise to the ‘unaided clinical judgement’ the GPs decided on which symptoms or findings they based their judgement on in a particular case. This test strategy is referred to as ‘aided clinical judgement’. All study physicians were blinded to the results of the reference standard.

Reference standard

It was anticipated that most patients would have a low probability of CHD and that invasive testing would not be allowed for ethical reasons in these patients. A panel diagnosis is recommended as an acceptable and valid alternative when a single reference test is not applicable.6–8 All patients included in the study were contacted by phone after 6 weeks and again at 6 months, and asked about their chest pain, further medical consultations, and treatments including drugs or hospitalisations. Additionally, their GPs were contacted — and specialists and hospitals if referred — to obtain relevant information about further consultations, diagnostic procedures, treatments, and discharge letters. An independent expert panel consisting of at least one GP and one research staff member reviewed each patient’s data and used recommended criteria9–11 to decide whether CHD had been the underlying cause for chest pain. Audits and thorough follow-up guaranteed that all relevant data were available to these experts. The experts resolved disagreements by discussion.

An assumption of this design is that CHD would manifest within the follow-up. As this reference standard is based predominantly on follow-up data, patients were counted as ‘loss to follow-up’ if they could not be reached by phone and the GP had no relevant data after 6 months. If data were available, but the expert panel found no conclusive diagnosis, cases were accordingly counted as ‘inconclusive’. The expert panel was not blinded to the results of the index tests.

Statistical analyses

Study physicians’ three-level rating of CHD probability (‘low probability’ = test negative, ‘intermediate’ or ‘high probability’ = test positive) were dichotomised. For each test strategy sensitivity, specificity, predictive values, and likelihood ratios were calculated. Lower and upper limits of confidence intervals (CIs) were calculated for proportions using the Wilson procedure without a correction for continuity12 and for likelihood ratios using the procedure recommended by Simel and colleagues.13

For each comparison between the established diagnostic test, the ‘unaided clinical judgement’, and one of the new test strategies (‘MHS’, ‘aided clinical judgement, or ‘triage’), the differences between each strategy’s sensitivity and specificity were calculated, respectively. To compare the accuracy of the ‘MHS’ and the ‘triage’ strategy with that of the ‘unaided clinical judgement’, a paired design was used. Each test strategy was independently applied to the same group of patients (subsample A). Calculations of differences needed to account for the paired nature of the data.14 However, it was impossible to apply the ‘aided clinical judgement’ and the ‘unaided clinical judgement’ independently to the same sample of patients. Therefore an unpaired design was used applying and comparing the accuracy of these test strategies in two different samples (subsamples A and B).

To illustrate the possible effects of using the different strategies the total number of true positives, false positives, true negatives, false negatives, and overall test positives were modelled in an imaginary population of 1000 patients assuming that the overall prevalence of CHD in this population would be the same as in this study. These numbers and the respective CIs were calculated for each test strategy using the sensitivity, specificity, test-positive fraction, and their respective CIs as obtained from the study data. Calculations were performed using R 2.15.0.

RESULTS

Figure 1 illustrates the flow of patients. In total, 832 patients were included in the current analysis.

Figure 1.
Figure 1.

Flow of patients.

MHS = Marburg Heart Score. CHD = coronary heart disease.

aGPs were not aware of the MHS results. bGPs were aware of the MHS results.

The reference diagnosis was CHD in 92 (11.1%) patients. The 39 study physicians unaware of the MHS recruited 578 patients (subsample A), while the 17 study physicians who did integrate the MHS into their judgement recruited 254 patients (subsample B). Table 2 shows study physicians’ and patients’ characteristics. It is important to note that, due to the low overall prevalence of CHD in general practice, changes in sensitivity are accompanied by only slight changes in absolute numbers of true positives and false negatives, respectively. In comparison, changes in specificity are accompanied by more distinct changes in absolute numbers of true positives and false negatives, respectively.

View this table:
Table 2.

Characteristics of GPs and patients

Table 3 shows the sensitivity, specificity, likelihood ratios, and predictive values for each test strategy, while Figure 2 shows the differences in the sensitivity and specificity for each comparison between one of the new strategies and the established test strategy. Compared with the ‘unaided clinical judgement’, the ‘MHS’ had a similar specificity but a higher sensitivity, while the aided clinical judgement had both, a higher specificity and sensitivity. These differences were not significant. The ‘triage’, when compared with the ‘unaided clinical judgement’, had a comparable sensitivity, but a significant higher specificity.

View this table:
Table 3.

Diagnostic accuracy of the four test strategies

Figure 2.
Figure 2.

Accuracy of the unaided clinical judgement compared with the accuracy of the three new test strategies that are based on the Marburg Heart Score.

Δ Sensitivity = difference in sensitivity compared with unaided clinical judgement.

Δ Specificity = difference in specificity compared with unaided clinical judgement.

Table 4 shows the absolute numbers of true positives, false positives, true negatives, false negatives, and overall test positives, assuming that all four test strategies were applied to an imaginary population of 1000 chest pain patients with a prevalence or pretest probability of CHD of 11%.

View this table:
Table 4.

Absolute numbers of true positives, false negatives, true negatives, false positives, and test positives in an imaginary population of 1000 patients with an 11% prevalence of CHD

DISCUSSION

Summary

This study showed that using the MHS as a triage test can improve the accuracy of the GP’s clinical judgement with a similar sensitivity and a significantly higher specificity. There was also a trend indicating that integrating the MHS into clinical reasoning (aided clinical judgement) may improve both sensitivity and specificity.

Strengths and limitations

Audits guaranteed the consecutive recruitment, and comprehensive collection of relevant follow-up data reduced the potential for misclassification. Both resulted in a small number of losses to follow-up, cases with missing values, and cases with an inconclusive diagnosis that may otherwise compromise the validity of the study.

This study also has limitations. It was performed in one area of Germany and the generalisability of the results to other national healthcare systems may be limited. The expert panel was not blinded towards the baseline data, including the index test results. Knowing the results of the index tests could have influenced the decision of the expert panel increasing the risk of an incorporation bias. If the expert panel had been blinded, fewer data for decision-making would have been available. This may have resulted in a misclassification bias and a higher rate of cases with an inconclusive reference diagnosis. The study physicians were not randomly assigned to one of the two study groups and there were differences in the location of the practice, and the proportion of patients with a history of smoking and hypertension. This may have compromised the unpaired comparison between the ‘unaided’ and ‘aided clinical judgement’. The calculation of the overall sample size aimed to allow a precise estimation of the measures of the MHS’s diagnostic accuracy.5 The study was also not powered to investigate the differences between the respective strategies.

Comparison with existing literature

Only a few clinical prediction rules for CHD have been externally validated in general practice.1,4,15 To the authors’ knowledge, this is the first study that investigated how one of these clinical decision rules adds to the GP’s judgement. This study gave some insights into how different ways of using the MHS may affect the accuracy of the initial diagnostic approach to chest pain patients.

The performance of the MHS was at least equal to that of the physicians’ unaided clinical judgement. Considering the simplicity of the MHS, this result is remarkable. Evidence was found that using the MHS as a triage test can improve the specificity of the initial assessment. A disadvantage of this formalised use of the MHS as a triage test may be that the MHS considers only one of the relevant causes of chest pain. Integrating the MHS into clinical reasoning in an informal way probably better reflects how clinicians actually use such rules. This approach resulted in an improvement of both sensitivity and specificity. These differences were not significant. However, an adequately powered (power = 80%) study that aims to detect a difference of 8% in sensitivity between the ‘aided’ and ‘unaided clinical judgement’ using an unpaired design and assuming a prevalence of 11% would require about 2500 patients in each group.

Implications for practice

When comparing different test strategies one must consider the clinical situation and the respective consequences of different test results for patient management. In the early workup of the patient with chest pain, a negative test result would imply the need to consider reasons other than CHD for chest pain. Consequently, a false-negative result may lead to harm from a delayed or missed diagnosis. On the other hand, the consequence of a positive test result would be further diagnostic workup for CHD. Thus, a false-positive result may lead to anxiety and morbidity from additional testing. It can be assumed that the potential harm of the former outweighs that of the latter.16

In addition, one must also consider different perspectives. From the perspective of a single patient, a high probability of not having the disease given a negative test result, the negative predictive value (NPV), is highly desirable. What is high enough may depend on the patient’s preferences. However, all test strategies showed similar and reasonably high NPVs ranging from 96.3% (unaided clinical judgement) to 98.7% (aided clinical judgement). There was also a trend indicating that the triage strategy results in a higher positive predictive value; that is, the probability of having the disease given a positive test result.

From the health system’s perspective, a high sensitivity and a high specificity are desirable. Compared with the unaided clinical judgement, using the triage strategy would result in a comparable absolute number of true positives and a significantly higher number of true negatives. As a result, a significantly lower number of patients without CHD would undergo unnecessary tests. This would be accompanied by a decreased number of test-positive patients who would need further diagnostic workup, indicating that this test strategy may also help reduce costs for further testing.

Clinicians should be aware that the MHS was developed and validated to assess patients presenting with chest pain. Studies conducted in various settings suggested that 7–20% of patients with myocardial ischaemia have no symptoms at all or present exclusively with atypical symptoms like dyspnoea or fatigue, and that this proportion is increased among older patients and patients with diabetes.17–20 It is not known how well the MHS would perform in these patients.

Ebell stated that, besides accuracy and the potential to improve patient-related outcomes, useful clinical decision rules should be transparent, easy to use, and should have a good face validity.21 The ‘any three of five’ rule of the MHS could hardly be easier to use and fits well into the daily routine of busy clinicians. The criteria used in the rule have a high face validity, leading to a good acceptance of the rule.22 It can also be assumed that many GPs already use at least some of them to base their judgement on. However, the advantage of the MHS is the standardised combination of these criteria, leading to greater transparency.

Acknowledgments

We thank all participating study physicians for their cooperation. We also would like to thank Juliette Rautenberg for providing English-language editing, improving the precision and fluency of the manuscript.

Notes

Funding

This study was funded by the Federal Ministry of Education and Research (BMBF — grant no. FKZ 01GK0701). The funding source had no involvement in the study.

Ethical approval

The study protocol was approved by the Ethics Committee of the Faculty of Medicine, Philipps University of Marburg, Germany. All participants gave informed consent before taking part.

Provenance

Freely submitted; externally peer reviewed.

Competing interests

The authors have declared no competing interests.

Discuss this article

Contribute and read comments about this article: bjgp.org/letters

  • Received January 23, 2015.
  • Revision requested February 17, 2015.
  • Accepted March 23, 2015.

REFERENCES

  1. 1.↵
    1. Sox HC Jr.,
    2. Hickam DH,
    3. Marton KI,
    4. et al.
    (1990) Using the patient’s history to estimate the probability of coronary artery disease: a comparison of primary care and referral practices. Am J Med 89(1):7–14.
    OpenUrlCrossRefPubMed
  2. 2.
    1. Verdon F,
    2. Herzig L,
    3. Burnand B,
    4. et al.
    (2008) Chest pain in daily practice: occurrence, causes and management. Swiss Med Weekly 138(23–24):340–347.
    OpenUrl
  3. 3.↵
    1. Bösner S,
    2. Becker A,
    3. Haasenritter J,
    4. et al.
    (2009) Chest pain in primary care: epidemiology and pre-work-up probabilities. Eur J Gen Pract 15(3):141–146.
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. Bösner S,
    2. Haasenritter J,
    3. Becker A,
    4. et al.
    (2010) Ruling out coronary artery disease in primary care: development and validation of a simple prediction rule. CMAJ 182(12):1295–1300.
    OpenUrlAbstract/FREE Full Text
  5. 5.↵
    1. Haasenritter J,
    2. Bösner S,
    3. Vaucher P,
    4. et al.
    (2012) Ruling out coronary heart disease in primary care: external validation of a clinical prediction rule. Br J Gen Pract doi:10.3399/bjgp12X649106.
    OpenUrlAbstract/FREE Full Text
  6. 6.↵
    1. Knottnerus JA,
    2. Buntinx F
    1. Knottnerus JA,
    2. Muris JW
    (2009) in The evidence base of clinical diagnosis: theory and methods of diagnostic research, Assessment of the accuracy of diagnostic tests: the cross-sectional study, eds Knottnerus JA, Buntinx F (Wiley-Blackwell Publishers/BMJ Books, Oxford, Hoboken, NJ), 2nd edn, pp 42–63.
  7. 7.
    1. Reitsma JB,
    2. Rutjes AWS,
    3. Khan KS,
    4. et al.
    (2009) A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. J Clin Epidemiol 62(8):797–806.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Bertens LC,
    2. Broekhuizen BD,
    3. Naaktgeboren CA,
    4. et al.
    (2013) Use of expert panels to define the reference standard in diagnostic research: a systematic review of published methods and reporting. PLoS Med 10(10):e1001531.
    OpenUrlCrossRefPubMed
  9. 9.↵
    1. Alpert JS,
    2. Thygesen K,
    3. Antman E,
    4. et al.
    (2000) Myocardial infarction redefined — a consensus document of the Joint European Society of Cardiology/American College of Cardiology Committee for the redefinition of myocardial infarction. J Am Coll Cardiol 36(3):959–969.
    OpenUrlFREE Full Text
  10. 10.
    1. Fox K,
    2. Garcia MA,
    3. Ardissino D,
    4. et al.
    (2006) Guidelines on the management of stable angina pectoris: executive summary: the Task Force on the Management of Stable Angina Pectoris of the European Society of Cardiology. Eur Heart J 27(11):1341–1381.
    OpenUrlCrossRefPubMed
  11. 11.↵
    1. Skinner JS,
    2. Smeeth L,
    3. Kendall JM,
    4. et al.,
    5. NICE guidance
    (2010) Chest pain of recent onset: assessment and diagnosis of recent onset chest pain or discomfort of suspected cardiac origin. Heart 96(12):974–978.
    OpenUrlFREE Full Text
  12. 12.↵
    1. Newcombe RG
    (1998) Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med 17(8):857–872.
    OpenUrlCrossRefPubMed
  13. 13.↵
    1. Simel DL,
    2. Samsa GP,
    3. Matchar DB
    (1991) Likelihood ratios with confidence: sample size estimation for diagnostic test studies. J Clin Epidemiol 44(8):763–770.
    OpenUrlCrossRefPubMed
  14. 14.↵
    1. Hayen A,
    2. Macaskill P,
    3. Irwig L,
    4. et al.
    (2010) Appropriate statistical methods are required to assess diagnostic tests for replacement, add-on, and triage. J Clin Epidemiol 63(8):883–891.
    OpenUrlCrossRefPubMed
  15. 15.↵
    1. Gencer B,
    2. Vaucher P,
    3. Herzig L,
    4. et al.
    (2010) Ruling out coronary heart disease in primary care patients with chest pain: a clinical prediction score. BMC Med 8(1):9.
    OpenUrlCrossRefPubMed
  16. 16.↵
    1. Knuuti J,
    2. Bengel F,
    3. Bax JJ,
    4. et al.
    (2014) Risks and benefits of cardiac imaging: an analysis of risks related to imaging for coronary artery disease. Eur Heart J 35(10):633–638.
    OpenUrlCrossRefPubMed
  17. 17.↵
    1. Mehta RH,
    2. Rathore SS,
    3. Radford MJ,
    4. et al.
    (2001) Acute myocardial infarction in the elderly: differences by age. J Am Coll Cardiol 38(3):736–741.
    OpenUrlFREE Full Text
  18. 18.
    1. Kannel WB,
    2. Abbott RD
    (1984) Incidence and prognosis of unrecognized myocardial infarction. An update on the Framingham study. New Engl J Med 311(18):1144–1147.
    OpenUrlCrossRefPubMed
  19. 19.
    1. Bhatia LC,
    2. Naik RH
    (2013) Clinical profile of acute myocardial infarction in elderly patients. J Cardiovasc Dis Res 4(2):107–111.
    OpenUrlCrossRefPubMed
  20. 20.↵
    1. Paul SD,
    2. O’Gara PT,
    3. Mahjoub ZA,
    4. et al.
    (1996) Geriatric patients with acute myocardial infarction: cardiac risk factor profiles, presentation, thrombolysis, coronary interventions, and prognosis. Am Heart J 131(4):710–715.
    OpenUrlCrossRefPubMed
  21. 21.↵
    1. Ebell M
    (2010) AHRQ White Paper: Use of clinical decision rules for point-of-care decision support. Med Decis Making 30(6):712–721.
    OpenUrlCrossRefPubMed
  22. 22.↵
    1. Kramer L,
    2. Rabanizada N,
    3. Haasenritter J,
    4. et al.
    (2011) Do guidelines on first impression make sense? Implementation of a chest pain guideline in primary care: a systematic evaluation of acceptance and feasibility. BMC Fam Pract 12:128.
    OpenUrlPubMed
Back to top

In this issue

British Journal of General Practice: 65 (640)
British Journal of General Practice
Vol. 65, Issue 640
November 2015
Download PDF
Download PowerPoint
Article Alerts
Email Article
Citation Tools

Chest pain for coronary heart disease in general practice: clinical judgement and a clinical decision rule
Jörg Haasenritter, Norbert Donner-Banzhoff, Stefan Bösner
British Journal of General Practice 2015; 65 (640): e748-e753. DOI: 10.3399/bjgp15X687385
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo

Jump to section

Keywords

Intended for Healthcare Professionals

 
 

British Journal of General Practice