Methods

We conducted a cohort study of children age 3 months–16 years attending the Paediatric Assessment Unit (PAU) at the University Hospital Coventry and Warwickshire NHS Trust (UHCW). Children were referred by general practitioners (GPs), self-referred or brought in by ambulance. Children were recruited to the study if the parents, referring clinician, or triage nurse suspected acute infection. We excluded children with diseases liable to cause repeated serious bacterial infection (including haematological malignancies, iatrogenic immunosuppression), and infections resulting from penetrating trauma. All children attending the PAU were triaged by a nurse on arrival. This assessment included identifying the presenting complaint, measurement of vital signs and conscious level (AVPU score), together with the MTS.15 The MTS system assigned children to four categories based on the maximum delay before further assessment: emergency (0 minutes), very urgent (10 minutes), urgent (60 minutes) and standard or non-urgent (120 minutes). The triage nurses assessed activity level, respiratory distress and hydration. The vital signs measured were axillary temperature (WelchAllyn SureTemp Plus, WelchAllyn, San Diego, CA, USA), heart rate and oxygen saturations (Nellcor N20E pulse oximeter, Nellcor, Pleasanton, CA, USA), respiratory rate (clinical count) and capillary refill time (CRT). A parental questionnaire was completed by parents on arrival at the PAU, which included a check list of 22 presenting symptoms. The children’s clinical features of colour, activity level, respiratory effort, hydration, presence of neck stiffness and non-blanching rash, as well as vital signs were categorised, blind to final outcome, into the NICE traffic light classification of intermediate (amber) and high (red) risk categories as detailed in the NICE guideline on management of the feverish child1 (see online supplementary table). Details of hospital admissions were obtained from the hospital medical records. For children who were either not admitted or admitted for less than 24 hours, we looked at PAU records for evidence of another visit in the next seven days.

Reference standard

We created a “severity of infection” reference standard based on the final diagnosis made by senior paediatricians at the time of discharge from the PAU, or inpatient ward if the child was admitted. The final diagnosis was categorised by the severity of infection: (1) minor infection—conditions from which the child was expected to recover without sequelae; (2) serious infection—conditions that were likely to be life-threatening if untreated or with high chance of life-threatening complications or sequelae; (3) intermediate infection—conditions that were not likely to be life-threatening, but were expected to last for longer than 10 days or have a non life-threatening complication; (4) not infection group—a final diagnosis that was not an acute infection. We compared the severity of infection standard with the need for hospital admission, length of stay and use of hospital-based interventions (intravenous fluids intravenous antibiotics, other parenteral medication, oxygen support, admission to high dependency unit, transfer to tertiary hospital). We found a strong correlation between diagnoses classed as severe infections and admission to hospital for more than 24 hours, which in turn correlated with the use of hospital-based interventions. We therefore decided to use the severity of infection as our reference standard (but the results on hospital admission are available, see online supplementary tables).

Data entry and statistical analysis

We tested associations between vital signs with severity of infection using χ² tests (or Fisher’s exact test where appropriate). Fever was defined as ⩾39.0°C, tachypnoea was defined as a respiratory rate that exceeded Advanced Paediatric Life Support (APLS) cut-offs (<1 year, 40 breaths/min; 1–2 years, 35 breaths/min; 3–4 years, 30 breaths/min; 5–12 years, 25 breaths/min), decreased oxygen saturations as ⩽94% and capillary refill time (CRT) as >2 seconds.16 For the purpose of analysis we decided to define tachycardia by two methods: first, as a heart rate that exceeded APLS cut-offs (<1 year, 160 beats/min; 1–2 years, 150 beats/min; 3–4 years, 140 beats/min; 5–12 years, 120 beats/min) and, second, we assessed tachycardia using the heart rate adjusted for temperature and age that exceeded the 90th centiles in children <11 years of age.14

We determined the combination of vital signs that provided optimum discrimination between serious and minor infection. The diagnostic characteristics of the MTS were dichotomised as (1) “standard” vs “urgent/very urgent/emergency” and (2) “standard/urgent” vs “very urgent/emergency”. Analyses were performed using SPSS software version 14.0 and the Catmaker software for diagnostic tests (http://www.cebm.net).

Results

A total of 700 children entered the study, after the exclusion of 42 because of treatment for cancer or transplant (17), admission more than once during study period (11), transfer to the PAU from another specialty (5), a chronic medical problem (3), incomplete records (3) and age <3 months (3). The median age was 3 years 1.5 months and 377 (53.9%) were male. The majority of children were from either white (512, 73.1%) or Asian (87, 12.4%) ethnic groups. Just over half (357, 51.0%) were referred from primary care (of whom 267 were referred from GP daytime surgery and 90 from the out-of-hours centre), 198 (28.3%) children were self-referrals, 116 (16.6%) were brought in after calling emergency ambulance and the remaining 29 (4.1%) were referred by NHS direct (9), referred by community or ward nurse (7), or were not recorded (13). Nearly two-thirds (460 or 65.7%) of the children had no significant medical history. In the remainder, asthma was the most frequent medical condition (57 children), followed by prematurity (40) and history of febrile seizure (24).

Overall, 383 (54.7%) children were admitted from the PAU (median length of stay 2 days). Thirty of these children were evaluated at the PAU for a second time within 7 days, of whom five were admitted (final diagnoses: mastoiditis, seizure of unknown cause, upper respiratory tract infection, fever of unknown cause and pneumonia).

Final diagnoses

A total of 339 (48.4%) children had a final diagnosis of minor infection, 108 (15.4%) serious infection, 205 (29.3%) intermediate infection and 48 (6.9%) did not have an infection although initially suspected of having one (table 1).

Association between vital signs and severity of infection

Temperature was recorded in 690 (98.6%) children, heart rate in 689 (98.4%), respiratory rate in 595 (85.0%), oxygen saturations in 672 (96.0%), capillary refill time in 392 (56.0%) and blood pressure in 109 (15.6%). Frequency of abnormal vital signs was significantly associated with severity of infection (table 2). Children with serious infection were significantly more likely than those with minor infections to have a temperature ⩾39°C (p<0.001), tachycardia (p<0.001), tachypnoea (p = 0.002), oxygen saturations ⩽94% (p<0.001) and CRT >2 seconds (p<0.001). Children with intermediate infection were significantly more likely than those with minor infection to have a temperature ⩾39°C (p = 0.004), tachycardia (p<0.001), oxygen saturations ⩽94% (p = 0.007) and CRT >2 seconds (p = 0.001). However, only tachypnoea (p = 0.009) and saturations ⩽94% (p = 0.001) differed significantly between children with serious and intermediate infections.

Children with serious or intermediate infection were significantly more likely than children with minor or no infection to have a temperature ⩾39°C (p<0.001), tachycardia (p<0.001), oxygen saturations ⩽94% (p<0.001) or CRT >2 seconds (p<0.001) (tachypnoea was of borderline significance p = 0.05). Although individual vital signs were poorly sensitive at discriminating children with serious or intermediate infection from those with minor or no infection (table 3), several were highly specific—namely, temperature ⩾39°C (87%), saturations ⩽94% (93%) and a CRT of longer than 2 seconds (100%). We found similar results when comparing the sensitivity and specificity of vital signs for discriminating children who had serious infection vs those with intermediate, minor or no infection (see online supplementary tables).

Comparison between diagnostic value of vital signs, with MTS and NICE clinical risk factors

The combinations of abnormal vital signs providing the optimum sensitivity and specificity were explored in the 527 children who had records for all four of temperature, heart rate, respiratory rate and saturations (93 serious infection, 155 intermediate infection, 279 minor infection and 41 not infection). We excluded CRT from the possible combinations as it was only measured in 56% of children. The presence of one or more of a temperature ⩾39°C, saturations ⩽94%, tachycardia and tachypnoea was 80% sensitive and 39% specific for discriminating serious/intermediate vs minor/no infection (table 4). This combination was 86% sensitive and 34% specific for discriminating serious from intermediate/minor/no infection (see online supplementary tables).

An MTS category was available for 668 (95.4%) children, of whom 187 (28.0%) were triaged as “standard”, 381 (57.0%) “urgent”, 97 (14.5%) “very urgent” and 3 (0.4%) as “emergency”. Children with an MTS category of “urgent, very urgent or emergency” were significantly more likely to have a serious infection than minor infection (95/107 (88.8%) vs 204/322 (63.4%), p<0.001) and an intermediate infection rather than minor infection (159/197 (80.7%) vs 204/322 (63.4%), p<0.001). This MTS threshold was highly sensitive (84–89%) but not specific (31–38%) for determining severity of infection (table 4). Using a more stringent threshold, children with an MTS category of “very urgent or emergency” were significantly more likely to have a serious infection than a minor infection (27/107 (25.2%) vs 22/322 (6.8%), p<0.001), and more likely to have an intermediate infection than a minor infection (47/197 (23.9%) vs 22/322 (6.8%), p<0.001). This threshold for the MTS was poorly sensitive (24–25%) but highly specific (87–93%) for serious or intermediate infection.

Using the NICE “traffic light” system, children within each infection group who had intermediate (“amber”) and high (“red”) risk clinical features are shown in table 5. This shows that amber or red features were more frequent in children with serious or intermediate infections than in those with minor infections. However, approximately three-quarters of children with minor infection had at least one amber or red feature noted. In contrast, almost one-quarter (26% or 24.1%) of children with serious infection had no red features present, and eight of them (7.4%) had no amber or red features present. The presence of one or more amber or red features was moderately sensitive (85%) for discriminating children with serious or intermediate infection from those with minor or no infection but had limited specificity (29%) (table 4)

Using tachycardia defined using new centile charts

Using centile charts to adjust heart rates for age and temperature in children under 11 years of age,14 we found that children with serious infection were significantly more likely to have heart rates above the 90th centile than children with minor infection (23/81 (28.4%) vs 35/281 (12.5%), χ² p = 0.001) (see online supplementary table). In addition, children with intermediate infection were more likely to have heart rates above the 90th centile than those with minor infection (42/174 (24.1%) vs 35/281 (12.5%), χ² p = 0.001). A heart rate exceeding the 90th centile was not sensitive (24%, 95% CI 19% to 29%) but was highly specific (88%, 95% CI 84% to 91%) for serious infection compared to minor infection (LR+ 1.94, 1.37–2.74), LR− 0.87, 0.81–0.94).

Discussion

Principal findings

Our study shows, as expected, that children presenting to a paediatric assessment unit who have more serious infections are more likely to have abnormal vital signs than those with minor infections. Nevertheless, the discriminatory value of individual vital signs is poor, and none offers sufficient sensitivity to be used alone. However, a temperature ⩾39°C, saturations ⩽94% and prolonged CRT are highly specific for serious or intermediate infections, suggesting that when these are present they are useful at “ruling in” serious infection. APLS thresholds for heart rate and respiratory rate have clinically irrelevant sensitivity and specificity.16 However, using tachycardia adjusted for age and temperature centiles, a heart rate above the 90th centile for age and temperature was highly specific for serious or intermediate infection, but was not sensitive, so again it would be useful at “ruling in” serious or intermediate infection.14

The limited diagnostic value of individual vital signs is not surprising since a uniform pathophysiological disturbance in all serious infections is unlikely. Clinically, several vital signs are measured and interpreted together. Indeed, we found that the presence of one or more of fever, tachycardia, tachypnoea and decreased oxygen saturations was moderately sensitive (80%) for identifying children with serious or intermediate infection, but with limited specificity (39%).

Comparison with previous research

Previous studies examining associations between individual vital signs and particular diseases have shown variable results. High temperature is associated in some studies with an increased risk of bacteraemia, and the association between tachypnoea and pneumonia is reasonably well established.17 18 19 20 21 Several paediatric early warning systems have been derived using combinations of physiological parameters in children.22 23 24 25 26 These have predominantly attempted to predict critical illness in hospitalised children. A score derived in an emergency department showed a sensitivity of 70% for identifying children requiring intensive care unit admission.22 Clearly these scoring systems have an important role in the early warning of critical illness, but their diagnostic performance is likely to be lower in less selected populations, or for predicting less critical outcomes. It is disappointing that the NICE traffic light system did not perform significantly better than the combined vital signs, as it adds features such as pallor and increased work of breathing.1 There are two reasons for this—first, the NICE system is targeted to a more limited age range (0–5 years) and, second, we were not able to identify from our standardised records all of the “amber” or “red” clinical features listed in the NICE guideline. Assessment of severity of illness based on a “global” impression or using scoring systems have previously been shown to have good predictive value for serious infections and are used in primary care.27 28 29 30 The MTS is designed as a triage system rather than predictor of particular outcome. Nevertheless, a recent study of the MTS in paediatric emergency care found a likelihood ratio of only 2.3 for identifying children with highly urgent medical illness, and was weak in triaging febrile children.31

Strengths and weaknesses

One of the strengths of this study is that we recruited children presenting with suspected acute infection, rather than those presenting with fever. We believe that this inclusion criterion is more reflective of everyday clinical practice where only a proportion of children with acute infection will present with fever, because many will have been given antipyretics before attendance, or because they have infections that are not always associated with high fever (for example, gastroenteritis, appendicitis). Our findings are likely to have external validity for similar clinical settings but we cannot determine the extent to which our findings would apply in other settings (for example, primary care), where the prevalence of serious infection is lower.

This study had several limitations. First, our severity of infection classification emphasised the clinical importance of a given infection. This reflects the changing epidemiology of serious infections in the UK where invasive bacterial pathogens are increasingly rare, so the majority of serious infections in this study were caused by pneumonia. Evidence to support our pragmatic classification comes from the close correlation between serious infection and hospital admission for more than 24 hours. Moreover, there was evidence of a gradient effect, with the intermediate group clearly lying midway between minor and serious in terms of abnormalities of vital signs. Second, recruitment of children was not as consecutive as we would have liked, with more recruitment occurring during periods when research nurses were present. Although we may have recruited a slightly higher proportion of children with more serious infection than expected, we did not find any other evidence of bias. Third, we used pragmatic cut-off points for vital signs rather than a formal clinical prediction rule. This makes our results more applicable to clinical practice, as many clinicians have difficulty interpreting and using more quantitative measures of test accuracy.32 Fourth, triage and clinical care was based on usual care, with the exception of measuring heart rate and temperature using defined devices. Frequency of measurement of other vital signs was variable, and the validity of respiratory rate and capillary refill time is questionable, based on previous evidence.7 8 9 10 11 Finally, comparison of the diagnostic accuracy of vital signs with that of the NICE traffic light system is somewhat limited as the NICE system was developed for a more limited age range (0–5 years), and because we did not have data available on all the “amber” and “red” clinical features.

Clinical and research implications

So, how should clinicians use vital signs in assessing children with acute febrile illnesses? Vital signs are just one component of the clinical assessment of an unwell child, and should add to the other features from the history and examination. We provide good evidence that in this setting several vital signs are individually highly specific for serious infection (temperature of 39°C and above, saturations ⩽94%, CRT >2 seconds, heart rate >90th centile for age/temperature) and if present, these would be useful at ruling in a more serious infection. The presence of one or more of a combination of vital signs to triage children provides moderate sensitivity in this setting. Moreover, this provides comparable sensitivity to more complicated systems such as the MTS or NICE traffic light system, and is more objective. Although sensitivity and specificity are not directly affected by disease prevalence, the diagnostic characteristics of vital signs would be different in a primary care population where the disease spectrum is markedly different from an acute paediatric unit. Applying the positive likelihood ratios that we found to a primary care population where fewer than 1% of children have serious infection would provide little change in the post-test probability of serious infection.27 What also still needs to be clarified is whether repeated measurement of vital signs could usefully detect a worsening clinical condition in children in the intermediate “amber” group of the NICE system. More accurate measurements of respiratory rate and CRT, and a more sophisticated severity score based on combined vital signs could develop this form of assessment.