Accuracy of symptoms and signs for coronary heart disease assessed in primary care

Participating GPs and patients

Only GPs who were prepared to undergo random recruitment audits were accepted. Participating practices had to recruit consecutively every patient attending who had chest pain either as the presenting complaint or on questioning. The recruitment period lasted 12 weeks for each practice. For logistical reasons, recruitment was staggered in four waves between October 2005 and July 2006.

Every patient aged over 35 years with pain localised in the area between the clavicles and lower costal margins and anterior to the posterior axillary lines was to be included. GPs were asked to recruit also at home visits and emergency calls. Patients were eligible irrespective of the acute or chronic nature of their complaint, or of previously known conditions including CHD or risk factors. Patients whose chest pains had subsided for more than 1 month, whose chest pains had been investigated already, and/or who came for follow-up of their chest pains were excluded. In emergency situations without sufficient time for patient information and written consent, relevant clinical items were documented and kept by the GPs. Later, for example after discharge from hospital, the patient was asked to participate in the study. The report form was only handed over to the study personnel if the patient gave consent.

Data collection

GPs took a standardised history and performed a physical examination according to a report form that was piloted and modified accordingly. Index tests covered first impression of the patient, duration and temporal patterns of pain, character, localisation and associated symptoms, known vascular diseases, risk factors, and relevant findings. GPs also recorded their preliminary diagnoses, investigations, and management related to the patients' chest pains. Patients were contacted by phone 6 weeks and 6 months after the index consultation. Study assistants blinded to the results of index tests asked about the course of their chest pain and treatments, including hospitalisations and drugs. Discharge letters from specialists and hospitals were requested from GPs if needed.

Precautions against selection bias

Participating GPs were recruited from a network of research practices associated with the Department of General Practice, University of Marburg. The importance of recruiting every patient with chest pain irrespective of the presumed likelihood of CHD was emphasised to GPs. GPs were visited at 4-week intervals to check report forms, recruitment logs, and compliance with study procedures. Random audits were performed by searching routine documentation of participating practices to identify cases of chest pain not included in the study.

Reference standard

As in most patients the probability of CHD would be low, an invasive reference standard, for example coronary angiography, was not considered to be ethically justified. After completed follow-up at 6 months, a reference panel of one cardiologist, one GP, and one research staff member of the Department of General Practice, University of Marburg reviewed each patient's data. For a ‘delayed-type reference standard’,20 they decided on the presence/absence of CHD or the need for urgent hospital admission at the time of patient recruitment (index consultation). This design is based on the assumption that serious diseases, such as CHD, would manifest themselves within the mentioned time period. Decision making by the panel was based on recommendations of the CHD guideline of the German programme for disease management guidelines.21

Patient history is part of the definition of acute and chronic CHD. However, providing the reference panel with clinical data recorded by GPs would have raised the possibility of incorporation bias. To reduce incorporation bias, the panel judged each patient first without the index test (that is, blinded to clinical data recorded by the GP, including preliminary diagnoses), only using the information gathered at follow-up: referred to as the ‘blinded reference standard’. In a second round, follow-up data were reviewed in a randomly changed order, together with history and findings recorded by GPs: referred to as the ‘unblinded reference standard’. In patients who could not or could only partly be reached for follow-up, GPs were contacted for relevant data. If sufficient data were available, the reference panel could still make a decision for these patients.

Statistical analyses

Sample size calculation was based on the primary research question with CHD as the reference criterion. In low-prevalence samples, the precision of estimates of sensitivity is critical. To establish a (low) sensitivity of 0.55 with a confidence interval (CI) of 0.45 to 0.65, 96 patients with CHD would be needed. Under the assumption that 8% of patients with chest pain had CHD, 1200 patients had to be recruited. This would allow estimation of high sensitivities with even more precision; for example, a 95% CI (with the same width for sensitivity = 0.95) would require only 19 patients with CHD.22

The main analysis is based on Sample I which excluded trauma cases and patients with data that were insufficient for a reference decision with regard to CHD (Figure 1): these were cases for analysis. A subpopulation of patients presenting with chest pain of less than 48 hours duration (Sample A) was also analysed for the alternative reference standard, that is, urgent hospital admission being indicated or not: these were acute cases.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Patient flow: from number of consultations to cases for analysis (Sample I) and acute cases (Sample A).

To reduce the number of analyses, index tests were analysed only if appropriate; for example, items related to GPs' general impression of what constitutes an acute case. For univariate analyses, sensitivities, specificities, positive and negative predictive values, positive and negative likelihood ratios (LRs), and diagnostic odds ratios (ORs) were calculated for all items covered by the report form.

To arrive at a small subset of criteria for clinical recommendation, those index test items that had P<0.05 (univariate analysis) and LRs indicating at least moderate diagnostic accuracy, that is, for inclusion LR ≥2, for exclusion LR ≤0.5 were selected. They were included as independent variables in multivariable logistic regression analysis. Known CHD, cerebrovascular diseases, and peripheral arterial disease were grouped into a combined variable, ‘clinical vascular disease’, which was positive if at least one of the single variables was positive. One cluster of logically related tests (time of the day) was not analysed further as only two of six items fulfilled univariate criteria.

Dependent variables were CHD and ‘indication for urgent hospital admission’. Variable selection was conducted using the backward stepwise procedure (P<0.05). ORs and 95% CIs were calculated for both patient groups. For the independent variable ‘worse with inspiration’, very unstable estimates for regression coefficients were obtained. By omitting this variable in the final model with all other variables selected so far; however, estimates for other variables were not affected.

Main analyses on diagnostic accuracy were based on the unblinded reference standard. The robustness of the results was tested in sensitivity analyses with the blinded reference standard. Agreement between blinded and unblinded reference standards was assessed by the κ statistic. All analyses were performed with SPSS software (version 14.0).

GP characteristics

A total of 209 GPs (GPs and general physicians) in Hessen, Germany were approached, of whom 35.4% agreed to participate in the study. Of the participating 74 GPs, 50 were male and 24 female. Most were situated in urban areas (63.5%); the mean age of GPs was 49 years (range 33 to 64 years).

Patient characteristics

A total of 1355 patients presenting with chest pain were approached by participating GPs; 99 refused to participate in the study and seven were excluded because they did not meet the inclusion criteria. Data were obtained on symptoms and signs for 1249 patients (Table 1). For the following analyses, patients with a history of trauma (n = 39), patients who dropped out of the study (n = 3), and those in whom no reference diagnosis could be made (n = 8) were excluded. Sample I included all cases for analysis (n = 1199), and Sample A included all acute cases (n = 349; Figure 1). Sixty patients were lost to follow-up and 11 died during the follow-up period but provided enough data to be included in Sample I.

Among patients without trauma, 180 (14.9%) were eventually given the reference diagnosis CHD and 70 patients (5.8%) had an indication for urgent hospital admission according to the reference panel. Of these 70 patients, 42 (60.0%) had ACS, 12 (17.1%) stable CHD, six (8.6%) hypertensive crisis, one (1.4%) acute heart failure, three (4.3%) cardiac arrhythmia, two (2.9%) pulmonary embolism, one (1.4%) obstructive airway disease, and one acute cholecystitis; two (2.9%) patients could not be classified.

Univariable analysis

Univariate results for all patients with the reference diagnosis CHD (acute and chronic cases) are listed in Appendix 1. Interestingly, character of pain, localisation and radiation, pain duration criteria, sex of patient, and cardiovascular risk factors, except for diabetes mellitus, were not useful indicators. However, age (male ≥55 years, female ≥65 years), pain getting worse on exercise, and the need for a home visit were useful indicators for CHD. Continuous pain, stinging pain, cough and signs of respiratory infection, pain depending on inspiration, localised muscle tension, and pain that was reproducible on palpation was associated with reduced likelihood of CHD.

Appendix 2 describes the subgroup of patients presenting with acute chest pain (<48 hours) against ‘indication for urgent hospital admission’ as the reference standard. As in the CHD group, most comorbidity factors were associated significantly with ‘indication for urgent hospital admission’. Age, hyperlipidaemia, diabetes mellitus, pain worse on inspiration and movement, patient assuming cardiac origin of pain, and the need for a home visit (home visit conducted by the GP because of the patient's chest pain) were further predictors. Five index tests of the cluster showed positive LRs >2: the GP's impression that ‘something is wrong with my patient’; pale; cold and clammy skin; patient unusually calm; and breathlessness.

Stinging pain, cough, pain depending on breathing or movement, muscle tenderness, and replicable pain on palpation all showed negative associations. No significant results were found for the different clusters of index tests for presentation and duration of a pain episode, frequency and time at onset of pain, pain localisation, or the index tests for nausea and vomiting and tightness.

Multivariable analysis

Items fulfilling the univariate selection criteria (listed in bold letters in Appendices 1 and 2) were selected for multivariable analysis, the results of which are reported in Table 2.

Clinical vascular disease, pain depending on exercise, patient assuming cardiac origin of pain, age, known heart failure, and diabetes mellitus were associated positively with CHD. Negative associations were found for stinging pain, cough present, localised muscle tension, and pain reproducible on palpation.

Pain radiating to the left arm, clinical vascular disease, home visit, and known heart failure were associated positively for the subgroup of patients with pain duration below 48 hours and the indication for urgent hospital admission. However, pain reproducible on palpation was a negative indicator for this group. For clinical use, Table 3 presents LRs of diagnostic items that were significant in the multivariable analysis for including or excluding CHD and ‘indication for urgent hospital admission’.

Blinded versus unblinded reference standard

As measures of agreement of blinded versus unblinded reference panel, κ = 0.61 (95% CI = 0.55 to 0.66) was obtained for CHD and κ = 0.54 (95% CI = 0.38 to 0.71) for ‘indication for urgent hospital admission’. Sensitivity analyses for accuracy comparing the unblinded and blinded reference standards were performed (results available on request). Although measures of diagnostic effectiveness are on average slightly reduced with the reference panel blinded to index tests results, they are of the same order of magnitude.

Summary of main findings

A large number of symptoms and signs, although traditionally regarded as useful, were not shown to be effective in this study. However, clinical vascular disease, pain depending on exercise, the patient assuming cardiac origin of pain, age, known heart failure, and diabetes mellitus were indicators of CHD. Pain being of a stinging character, reproducible on palpation, localised muscle tension, and the presence of cough were associated with reduced likelihood of CHD.

Strengths and limitations of the study

To the authors' knowledge, this is the largest study investigating the diagnostic accuracy of cardiac symptoms and signs in primary care. The diagnostic tests evaluated were chosen according to the results of an exploratory analysis of GPs' decision making in daily practice.19 Study procedures ensured consecutive patient recruitment in a large number of urban and rural practices. An independent reference panel reviewed diagnostic data. This is therefore a phase III diagnostic study, the results of which should have immediate relevance for clinical practice.23

Given the low prevalence of CHD, the authors think there is no alternative to the ‘delayed-type reference standard’, that is, an expert panel reviewing follow-up data. However, the panel often had to decide on the basis of limited data as there was no requirement for GPs to use defined investigations. This was especially the case in older patients, some of them with already known CHD, whom their GP did not investigate beyond a resting ECG. Even fewer data were available for assessment blinded to clinical data recorded by GPs (index tests). There is a trade-off between reducing incorporation bias by blinding, and providing sufficient information for the reference decision, that is, by providing comprehensive data including the results of index tests. As a result, the reference standard in the present study cannot be regarded as perfect.

Some item clusters, for example temporal pattern of pain, showed a higher rate of missing data than others because this kind of information was difficult to frame within given categories. Due to the smaller sample size (n = 350), the subanalysis of the acute cases produced less precise estimates than the main analysis and might therefore be more affected. Given the overall large number of analyses, type 1 error cannot be ruled out. Calculations with CHD as the reference are the study's main analyses, whereas those with an ‘indication for urgent hospital admission’ must be regarded as descriptive.

The study data reflect the German health system where patients with chest pain may directly contact the emergency service, the hospital, or a cardiologist. Without a registered list of patients it was not possible to find out how many patients were admitted to hospital during the study period.

Comparison with existing literature

A systematic review of acute and chronic chest pain presenting in primary care showed that single clinical features in isolation were not helpful in ruling in or out ACS. The most helpful clinical features were pleuritic pain and pain on palpation.24 These findings are supported by a diagnostic meta-analysis determining the accuracy of 10 important signs and symptoms in diagnosing acute myocardial infarction or ACS. Only chest wall tenderness on palpation largely ruled out acute myocardial infarction or ACS,18 a finding that is also supported by the present data. A clinical review of the value and limitations of chest pain history in the evaluation of patients with suspected ACS reached similar conclusions.25 Although the present study used the indication for acute hospital admission as a reference criterion, the results are comparable as ACS will make up the majority of these cases. The present study also adds the need for a home visit as an indicator for acute hospital admission.

In a systematic review about the accuracy of bedside findings for diagnosing CHD, Chun and McGee found age and history of prior myocardial infarction to be relevant predictors.26 This is also supported by the present findings.

In a study undertaken in an intensive care unit, pain localisation and radiation could not discriminate between patients with or without myocardial infarction.10 In a prospective study undertaken in an emergency ward, Berger et al reported that right arm involvement can help to differentiate coronary disease from chest pain of other origin.27 However, both studies were undertaken in high-prevalence settings. The findings of the present study show that in a low-prevalence setting neither the localisation nor the radiation of pain yield significant diagnostic accuracy for CHD. However, radiation to the left arm predicts the need for urgent hospital admission in patients with acute chest pain. The fact that in the findings of the present study neither the duration and frequency of pain, nor the pain character and the time at onset were found to be valuable predictors is in line with the above quoted studies.

Goodacre et al examined clinical predictors for ACS in patients presenting with acute undifferentiated chest pain in an emergency department. They identified male sex, burning pain, radiation in the right or left arm, vomiting, and smoking as significant and independent criteria.14 The same variables were examined in the present study, for a low-prevalence setting, and it was found that all except radiation in the left arm (for the subgroup ‘indication for urgent hospital admission’) were not effective.

The study analysis incorporated items that are unique to primary care; for example, whether a home visit was asked for, whether the patient assumed his or her heart to be the origin of pain, and whether the patient gave a different impression compared to previous visits. The former two proved to be effective diagnostic tests.

Implications for clinical practice

Several authors have deplored the dearth of studies on diagnostic accuracy,28 especially with regard to the history and physical examination.29 This study has found that most symptoms and signs should not have a role in the diagnosis of CHD. However, there remain a number of symptoms and signs that show significant diagnostic accuracy for both CHD and the ‘indication for urgent hospital admission’. Most of these LRs are in a range that goes along with a small to moderate change in disease likelihood.30 In isolation, they are not suitable to rule in or out disease. However, when taken together and applied stepwise following the Bayesian approach (see the example in Figure 2), they can guide decisions with regard to further work-up or referral.31

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Applying the recommendations (example).