Diagnosing community-acquired pneumonia via a smartphone-based algorithm: a prospective cohort study in primary and acute-care consultations

Background Community-acquired pneumonia (CAP) is an essential consideration in patients presenting to primary care with respiratory symptoms; however, accurate diagnosis is difficult when clinical and radiological examinations are not possible, such as during telehealth consultations. Aim To develop and test a smartphone-based algorithm for diagnosing CAP without need for clinical examination or radiological inputs. Design and setting A prospective cohort study using data from participants aged >12 years presenting with acute respiratory symptoms to a hospital in Western Australia. Method Five cough audio-segments were recorded and four patient-reported symptoms (fever, acute cough, productive cough, and age) were analysed by the smartphone-based algorithm to generate an immediate diagnostic output for CAP. Independent cohorts were recruited to train and test the accuracy of the algorithm. Diagnostic agreement was calculated against the confirmed discharge diagnosis of CAP by specialist physicians. Specialist radiologists reported medical imaging. Results The smartphone-based algorithm had high percentage agreement (PA) with the clinical diagnosis of CAP in the total cohort (n = 322, positive PA [PPA] = 86.2%, negative PA [NPA] = 86.5%, area under the receiver operating characteristic curve [AUC] = 0.95); in participants 22–<65 years (n = 192, PPA = 85.7%, NPA = 87.0%, AUC = 0.94), and in participants aged ≥65 years (n = 86, PPA = 85.7%, NPA = 87.5%, AUC = 0.94). Agreement was preserved across CAP severity: 85.1% (n = 80/94) of participants with CRB-65 scores 1 or 2, and 87.7% (n = 57/65) with a score of 0, were correctly diagnosed by the algorithm. Conclusion The algorithm provides rapid and accurate diagnosis of CAP. It offers improved accuracy over current protocols when clinical evaluation is difficult. It provides increased capabilities for primary and acute care, including telehealth services, required during the COVID-19 pandemic.


INTRODUCTION
Although CAP remains a leading cause of morbidity and mortality, an accurate diagnosis can be difficult and is reliant on excellent clinical skills with or without radiology. Diagnosis is more difficult in older people where symptoms and signs may be minimal, or where the presentation is modified by underlying comorbidities. 1 Moreover, as current guidelines recommend moving towards digital consultations during the COVID-19 pandemic, 2 diagnosing CAP is even more challenging when doctors are unable to conduct clinical examinations. There is a need to develop accurate methods to assist physicians to diagnose CAP without relying on clinical examination or radiology, with desirable features including ease of use and point-of-care results.
Globally, pneumonia is the most common cause of infectious mortality, with >2 million adults dying from lower respiratory infections in 2015. 3 CAP is an essential consideration in patients presenting with respiratory symptoms. In the UK, 5%-12% of patients presenting to primary care with respiratory conditions have CAP. 4 In the US, 80% of patients with CAP are managed as outpatients; 5 and >8 000 000 patients with CAP are admitted to hospital annually, with an overall mortality rate of 8.8%. 6 The diagnosis of CAP relies on the presence of select clinical features (cough, fever, sputum production, and pleuritic chest pain) and abnormal vital signs (temperature, breathing, and heart rates), supported by finding new infiltrates on chest X-rays (CXRs). Despite this, clinical symptoms and signs in isolation have generally performed poorly as diagnostic criteria. 7 Auscultation findings alone have poor sensitivity and demonstrate poor agreement between clinicians. [8][9][10] Vital sign measurements are recommended to rule out CAP: a systemic review in 2019 showed validity using a combination of breathing rate, heart rate, and fever (negative likelihood ratio 0.24, 95% confidence interval [CI] = 0.17 to 0.34). 11 The utility of CXRs in diagnosing CAP is questionable. Agreement on CXR interpretation between emergency department clinicians or GPs with radiologists is poor. [12][13][14] CXRs have been shown to lack precision, reliability, and consistency, and are not useful in determining disease aetiology. [15][16][17] More information can be obtained from computed tomography and ultrasound examinations, but they are costly and not readily available. In primary care Abstract Background Community-acquired pneumonia (CAP) is an essential consideration in patients presenting to primary care with respiratory symptoms; however, accurate diagnosis is difficult when clinical and radiological examinations are not possible, such as during telehealth consultations.

Aim
To develop and test a smartphone-based algorithm for diagnosing CAP without need for clinical examination or radiological inputs.

Design and setting
A prospective cohort study using data from participants aged >12 years presenting with acute respiratory symptoms to a hospital in Western Australia.

Method
Five cough audio-segments were recorded and four patient-reported symptoms (fever, acute cough, productive cough, and age) were analysed by the smartphone-based algorithm to generate an immediate diagnostic output for CAP. Independent cohorts were recruited to train and test the accuracy of the algorithm. Diagnostic agreement was calculated against the confirmed discharge diagnosis of CAP by specialist physicians. Specialist radiologists reported medical imaging.

Conclusion
The algorithm provides rapid and accurate diagnosis of CAP. It offers improved accuracy over current protocols when clinical evaluation is difficult. It provides increased capabilities for primary and acute care, including telehealth services, required during the COVID-19 pandemic. settings, access to radiology may be limited and take time, thus delaying the initiation of treatment.

Keywords
Guidelines for the role of CXR in CAP diagnoses are inconsistent. The American Thoracic Society recommends routine CXRs for suspected CAP. 18 The British Thoracic Society recommends CXRs for hospitalised patients but not for community treatment, unless there are additional concerns such as inadequate response to treatment. 19 The European Respiratory Society recommends separating 'definite' from 'suspected' CAP based on the presence or absence of an abnormal CXR; then treating both groups with empirical antibiotics. 20 Most CAPs in the UK are thus diagnosed using clinical features alone.
In 2015, the authors commenced a digital diagnostic program to develop algorithms that combine a mathematical analysis of cough-associated audio segments with patient-reported symptoms to identify respiratory diseases in children and adults. [21][22][23] The forced expiratory air column produced during a cough supports a higher bandwidth than that across the chest wall, which is relied on in traditional auscultation. Sounds generated inside the lungs propagate through this air column and the pathophysiological changes caused by different respiratory conditions modulate the sound quality. The identification of unique sound signatures characteristic of different conditions led to the development of algorithms for the diagnosis of each respiratory condition. The algorithms do not rely on the input of vital signs, clinical examination, or radiographs.
A pilot study using this method demonstrated a sensitivity of 94% and specificity of 88% for differentiating pneumonia from no disease. 24 Subsequently, the authors published the accuracy of an algorithm to differentiate paediatric patients with croup, asthma, pneumonia, bronchiolitis, and upper/lower respiratory disease. 21 The PPA and NPA for pneumonia in children aged 2-12 years were 85% and 80%, and for >28 days to 2 years, 100% and 97%. The aim of the current study was to develop and test the performance of an algorithm to diagnose CAP in an adolescent and adult population presenting to primary and acute-care settings with respiratory symptoms.

Study design
A prospective cohort study comparing the diagnostic accuracy of an automated algorithm to clinical diagnosis of CAP in participants aged >12 years attending acutecare units in an Australian hospital.

Study population and setting
Between January 2016 and March 2019, the study recruited two cohorts as convenience samples from a large, general hospital in Western Australia. Data from the first cohort were used to develop and optimise the software algorithm. The second, independent cohort was used to test the optimised algorithm.
Participants came from multiple hospital departments, including the primary care unit, emergency department, and inpatient wards. Participants were approached if they presented with signs or symptoms of acute respiratory disease. All participants who met the inclusion criteria were eligible, except for those with documented chronic obstructive pulmonary disease (COPD) or restrictive lung disease (diagnostic algorithms have already been developed for these by the authors). Inclusion/exclusion criteria are presented in Table 1.

Study protocol
The study did not interfere with clinical care. There were no adverse events reported.

Cough recording
Different operators collected cough recordings at the time of enrolment using iPhone 6 smartphones held 25 cm to 50 cm from the mouth out of direct air flow. Recordings occurred in typical environments with background noises (talking, medical devices, footsteps, and doors).
Participants provided five voluntary coughs. The procedure took <1 minute to perform. Recording coughs from other people or television sounds were avoided.

How this fits in
Diagnosis of CAP in the primary care setting relies upon the identification of clinical features or abnormal vital signs during a clinical examination. The authors have developed a smartphone-based algorithm that removes the requirement for in-person consultation and provides high-diagnostic agreement with specialist diagnosis of CAP. The algorithm requires the input of five cough-sound segments and four patient-reported symptoms and provides a result in less than one minute. With increasing momentum towards digital-first care under the NHS, tools such as this that allow remote deployment are likely to find increased merit.

Clinical data
Data collected included the treating physician's final diagnosis; the patient's demographics, medical history, presenting symptoms, vital signs, examination findings, response to treatments; and results of investigations performed.

Clinical examination
A full medical assessment was performed on all participants at time of enrolment. Table 2 shows the definition of CAP (as per European Respiratory Society 20 ). The medical record for each patient was reviewed by an independent physician who took into consideration all available laboratory/ radiology results and clinical examination findings to confirm a final clinical diagnosis. Pneumonia severity was assessed using the CRB-65. 25 After a clinical diagnosis was assigned the database was locked to further input.

Development of the algorithm (index test)
Between January 2016 and November 2017, the study recruited participants to train and refine the algorithm. The mathematical techniques used to derive the algorithms have been described elsewhere. 21,23,24 Briefly, selected cough audio-segments and patient-reported symptoms were extracted from the training cohort and combined into several continuous classifier models to determine the probability of CAP. A logisticregression model was trained to diagnose CAP. The optimal model and corresponding probability decision thresholds were selected using a receiver operating characteristic (ROC) curve with due consideration given to achieving a balance of PPA and NPA. 24 The algorithm represents the weighted combination of clinical and cough derived features used.

Diagnostic accuracy study of the optimised diagnostic algorithm
Participants from the same locations were recruited as the training set, using the same inclusion/exclusion criteria, for the prospective diagnostic accuracy trial. Automatic segmentation was used to extract five coughs for analysis. For each subject, the algorithm reached a decision using audiodata, plus input from four patient-reported symptoms: presence of fever in the past week, presence of either acute (≤7 days) cough or productive cough, and age. An independent operator ran the algorithm on the testing set to ensure blinding.

Statistical analysis
As a clinical diagnosis of CAP is a not a gold-standard reference, PPA and NPA were used as primary measures of diagnostic agreement, rather than sensitivity or specificity. PPA reports clinical diagnosispositive cases who are also index test positive, whereas NPA reports clinical diagnosis-negative cases who are also index test negative. Results that were unsure (reference test) or missing (index test) were excluded from analysis.

Table 2. Clinical diagnosis definition
Community-acquired pneumonia (CAP) Must have: • New (≤7 days) respiratory symptoms (shortness of breath, cough, chest pain) • Acute fever within the last week • Treating team diagnosis of CAP and treated with antibiotics includes: Radiology confirmed CAP: radiologist-reported new consolidation or significant infiltrate on the CXR or consolidation on CT.
Not radiologically confirmed CAP: either no CXR performed or no consolidation/infiltrate on CXR.
CT = computed tomography. CXR = chest X-ray. Power calculations were derived based on expected PPA and NPA >85% from the training program. A minimum of 48 cases were required. Ninety-five per cent CIs were calculated using the method of Clopper-Pearson. The probability of a positive clinical diagnosis was calculated by the final classifier model and was used as the decision thresholds in derived ROC curves with area under curve (AUC) calculations.
Results are shown for the entire cohort and the following age groups: 12-<22 years; ≥22 years; 22-<65 years, and ≥65 years. These age groupings are consistent with both Food and Drug Administration regulatory and CRB-65 guidelines. 25,26 The underlying codes are the property of ResApp Health and are not available. The datasets supporting the conclusion of this article are available on reasonable request from Paul Porter and Joanna Brisbane. The cough recordings are not available but will be uploaded as an educational tool in the future.

Diagnostic accuracy study
Between December 2017 and March 2019, 331 participants were enrolled and completed the index and reference tests. Nine participants were excluded because of recording issues or uncertain clinical diagnoses, leaving 322 participants: 159 with CAP and 163 with non-pneumonic respiratory disease (Figure 1). Of these, 200 came from the emergency department or inpatient wards and 122 from ambulatory care (data not shown).
Demographics and clinical features are shown in Tables 3 and 4. The mean age of all participants was 48.5 ±22.0 years, 61.5% were female. There were more females than males for participants aged ≥22 years (62.6% versus 37.4%, P<0.01). There were no differences in past medical history between the 22-<65 and ≥65 years age groups for chronic respiratory disease (P = 0.28) or smoking history (P = 0.97). Participants in the ≥65 years age group were more likely to have comorbidity chronic heart failure (P<0.01), or atopic history (P<0.001) than younger participants.

Table 3. Characteristics of participants included and excluded from the analysis. Data includes all subjects in each age group (CAP positive and negative)
In participants aged 12-22 years (n = 44), five had CAP. All five were correctly diagnosed, while 6 of 39 participants without CAP incorrectly tested positive (data not shown).
Despite the increasing prevalence of comorbidities (heart failure and smoking history) in participants aged ≥65 years with CAP (    Table S1).

DISCUSSION Summary
The authors have developed and tested a mathematical algorithm for diagnosing CAP using cough sound analysis and patientreported symptoms. The algorithm showed a high degree of agreement with clinician diagnosis (utilising all clinical and imaging modalities) with accuracy maintained across age groups and severity indices. The tool does not rely on the input of vital signs, clinical examination, or radiological findings, and delivers an immediate, point-of-care result.

Strengths and limitations
The study design ensured the independence of the training/testing sets and the clinical/ algorithm diagnostic teams. This study applied stringent criteria, utilising all available medical and imaging data, to determine a clinical diagnosis that was as accurate as possible. The reference diagnosis of CAP was confirmed after patient discharge, whereas the algorithm provided diagnosis at the time of presentation. The study implemented this rigorous process as studies have shown up to 30% discordance between initial and final diagnoses of pneumonia. [27][28][29] This study builds on the authors' work diagnosing pneumonia in children aged 1 month to 12 years (PPA 85%, NPA 80%). 21 The numbers in this study exceeded that required by power calculations, except in the 12-<22 years age group. Although there was good agreement between the algorithm and clinical diagnoses, and the authors have no reason to consider that diagnostic accuracy would not be maintained across this age group, it would be helpful to confirm this in a larger study.
This study population was from a firstworld metropolitan hospital. It will be important to replicate the study in general practice, digital, and other settings to assess diagnostic accuracy and user experience. As the algorithm is site and operatorindependent, the authors believe that the results of the study are generalisable to community use. It was encouraging to note the high rate of usable cough datasets.
The inclusion criteria did not distinguish between microbiological aetiologies of CAP.
Determining the organism, or organisms, responsible is difficult and unreliable in most settings, and, consequently, CAP is routinely treated with antibiotics. Using polymerase chain reaction testing, along with standard microbiological techniques, has shown that a majority of cases have >1 organism and that, in winter at least, mixed viral-bacterial infections are more common than pure bacterial or viral. 30 As the study was performed before the SARS-CoV-2 pandemic participants with this aetiology were not included.
A diagnostic concern in elderly patients presenting with respiratory symptoms is differentiating acute heart failure from CAP. The authors were pleased to see that the accuracy was preserved in the ≥65 years age group, which had a higher prevalence of chronic heart failure.
Judging the severity of CAP is crucial when managing CAP and can be determined using the CRB-65, as recommended in UK guidelines. 19 CRB-65 uses an assessment of mental state and vital signs (including blood pressure) to stratify the severity of pneumonia and 30-day mortality in hospitals, and suggest where management should occur. 25 Using the CRB-65, 40.8% of this study's CAP-positive cohort scored 0 (1.2% risk of 30-day mortality, suitable for community treatment) and 59.2% scored 1 or 2 (8.2% risk of 30-day mortality, hospital referral recommended). 31 The study found no differences in the diagnostic accuracy of the algorithm between these groups. As the study excluded patients with severe respiratory distress, the severity of CAP in this study represents the population seen in general practice despite being hospital sourced. The authors anticipate that it would be more difficult to clinically diagnose pneumonias of lower severity. The ability of this algorithm to identify lower severity CAP is encouraging for its potential use in primary care.

Comparison with existing literature
Many authors have reported on the difficulties of diagnosing CAP, and this is reflected in inconsistent management guidelines. Most have focussed on the relative merits of clinical features, examination findings, and CXR, with very few reporting on digital diagnostic tools. 32 The authors' prior work has shown that it is possible to identify cough-sound streams that are characteristic of different respiratory disorders, including lower versus upper airway disease, COPD, COPD exacerbations, and asthma exacerbations in adults, and focal pneumonia, croup, and bronchiolitis in children. 21 In the absence of radiology, GPs are reliant on empirical diagnosis of CAP using auscultation or clinical signs resulting in a significant proportion (71%) of radiologically-confirmed CAP cases

Funding
ResApp Health (RAP) provided funding to support the Breathe Easy Program at Joondalup Health Campus (JHC) and University of Queensland (UQ). JHC provided office space, IT services, and consumables in kind. being missed in primary care. 33 Clinical signs and symptoms have previously been shown to perform poorly in the diagnosis of CAP. 11,13 This study's algorithm provides high diagnostic accuracy in the absence of radiology and without the need to examine the patient.

Implications for practice
The NHS has advised it expects all patients to have the right to be offered digital-first primary care by 2023-2024 with estimates that up to one-third of face-to-face GP visits could be replaced by telehealth each year. 34 In April 2020, during the COVID-19 pandemic, 71% of all GP visits in the UK were conducted remotely, compared to 25% pre-pandemic. Over 30% of telehealth consultations are for respiratory disease. 35,36 During the pandemic, the National Institute for Health and Care Excellence recommended that clinicians should not consult face-to-face or perform auscultation for CAP diagnosis unless essential. Instead, they suggest visual observation for breathlessness and cyanosis, and using clinical 'gestalt' or vital signs (including temperature, and heart and respiratory rates) to rule out CAP. 37,2 This would be challenging over a telephone or video call.
The authors have demonstrated that their algorithm can accurately identify CAP of varying severity, without clinical or radiological inputs. Its rapid output and smartphone platform make it suited for both traditional and digital consultations where it can assist in CAP diagnosis using only cough sounds and four patient-reported symptoms.
The algorithm is incorporated into a platform to identify multiple respiratory disorders and may enhance the capabilities of community telehealth services, with implications for reducing the duration of illness, minimising complications, and promoting good antibiotic stewardship.
The platform has regulatory approval for use in Europe and Australia. It is available on Australian telehealth platforms.
With current recommendations to use digital consultations during the pandemic, as well as the uptake of remote consultations globally, this tool presents an opportunity to use digital diagnostics to enhance telemedicine consultations.

Provenance
Freely submitted; externally peer reviewed.