Diagnostic accuracy study
Between December 2017 and March 2019, 331 participants were enrolled and completed the index and reference tests. Nine participants were excluded because of recording issues or uncertain clinical diagnoses, leaving 322 participants: 159 with CAP and 163 with non-pneumonic respiratory disease (Figure 1). Of these, 200 came from the emergency department or inpatient wards and 122 from ambulatory care (data not shown).
Figure 1. The flow of participants through the diagnostic accuracy study. Index test = software algorithm; reference test = clinician diagnosis. CAP = community-acquired pneumonia.
Demographics and clinical features are shown in Tables 3 and 4. The mean age of all participants was 48.5 ±22.0 years, 61.5% were female. There were more females than males for participants aged ≥22 years (62.6% versus 37.4%, P<0.01). There were no differences in past medical history between the 22–<65 and ≥65 years age groups for chronic respiratory disease (P = 0.28) or smoking history (P = 0.97). Participants in the ≥65 years age group were more likely to have comorbidity chronic heart failure (P<0.01), or atopic history (P<0.001) than younger participants.
Table 3. Characteristics of participants included and excluded from the analysis. Data includes all subjects in each age group (CAP positive and negative)
Table 4. Symptoms and past medical history of participants. Data includes all participants in each age group (CAP positive and negative)
Participants aged 22–<65 years with CAP were more likely to have a history of intermittent asthma than their age matched participants without CAP (25.0% versus 9.2%, P = 0.002); however, there were no differences in smoking history (50.0% versus 38.9%, P = 0.22), comorbidities of heart failure (2.4% versus 0.0%, P = 0.1), or atopy (25.0% versus 18.4%, P = 0.21). More participants aged ≥65 years with CAP had a history of heart failure than their age matched participants without CAP (40.0% versus 0.0%, P = 0.001); however, there were no difference in smoking history (47.1% versus 31.3%, P = 0.28), chronic respiratory disease (21.5% versus 18.8%, P = 0.99), or atopy (8.6% versus 6.3%, P = 0.20) (data not shown).
Participants without CAP (n = 163) had a primary diagnosis of upper respiratory tract infection (n = 50), non-pneumonic lower respiratory infection including acute bronchitis not reaching CAP definition (n = 32), acute asthma (n = 23), chronic obstructive airway disease (n = 22), allergy (n = 21), and non-respiratory disease (n = 15) (data not shown).
The PPA and NPA of the algorithm with clinical diagnosis and AUC are shown in Table 5. The ROC curves are shown in Supplementary Figures S1a, S1b, and S1c.
Table 5. PPA and NPA and calculated AUC of the software algorithm compared with a clinical diagnosis of CAP
For the total cohort, PPA was 86.2% (95% CI = 79.8 to 91.1) and NPA was 86.5% (95% CI = 80.3 to 91.3). Accuracy was preserved across age groups: ≥65 years (PPA = 85.7%, NPA = 87.5%, AUC = 0.95) and 22–<65 years (PPA = 85.7%, NPA = 87.0%, AUC = 0.94).
Participants with CAP had a CRB-65 severity score of 0 in 40.8% (n = 65/159) and between 1–2 in 59.2% (n = 94/165). CAP severity did not affect accuracy: 85.1% (n = 80/94) of participants with CRB-65 scores 1–2, and 87.7% (n = 57/65) with a score of 0 were correctly diagnosed (data not shown).
In participants aged 12–22 years (n = 44), five had CAP. All five were correctly diagnosed, while 6 of 39 participants without CAP incorrectly tested positive (data not shown).
Despite the increasing prevalence of comorbidities (heart failure and smoking history) in participants aged ≥65 years with CAP (Table 4) the accuracy was not reduced compared to the younger, healthier subgroup of 22–<65 years.
A sensitivity analysis including only participants with an acute cough improved the PPA of the total cohort from 86.2% to 92.5% (95% CI = 86.6% to 96.3%); however, significantly decreased the NPA from 86.5% to 42.1% (95% CI = 26.3% to 59.2%) and the AUC decreased from 0.95 to 0.62 (95% CI = 0.30 to 0.95) (see Supplementary Table S1).