Abstract
Background The diagnostic value of alarm features of serious infections in low prevalence settings is unclear.
Aim To explore to what extent alarm features play a role in referral to the emergency department (ED) by GPs who face a febrile child during out-of-hours care.
Design and setting Observational study using semi-structured, routine clinical practice data of febrile children (<16 years) presenting to GP out-of-hours care.
Method Logistic regression analyses were performed to assess the association between alarm features of serious infections (selected from two guidelines and one systematic review) and referral to the ED. Adherence to the guideline was explored by a 2×2 contingency table.
Results In total 794 (8.1%) of 9794 eligible patients were referred to the ED. Alarm signs most strongly associated with referral were ‘age <1 month’, ‘decreased consciousness’, ‘meningeal irritation’, and ‘signs of dehydration’. Nineteen percent of 3424 children with a positive referral indication according to the guideline were referred to the ED. The majority of those not referred had only one or two alarm features present. A negative referral indication was adhered to for the majority of children. Still, in 20% of referred children, alarm features were absent.
Conclusion In contrast to guidance, GPs working in primary out-of-hours care seem more conservative in referring febrile children to the ED, especially if only one or two alarm features of serious infection are present. In addition, in 20% of referred children, alarm features were absent, which suggests that other factors may be important in decisions about referral of febrile children to the hospital ED.
INTRODUCTION
In primary care, GPs frequently encounter febrile children, who are at risk of serious infections, such as meningitis, sepsis and pyelonephritis,1,2 which can lead to morbidity and mortality.3–5 The combined prevalence of serious infections in primary care; however, is less than 1%.6 Therefore GPs have the challenging task of distinguishing between the majority of children who have a low risk of serious infection and the minority at high risk who require further action.
Studies on identifying serious infections in low-prevalence settings are scarce.6–9 Current clinical guidelines supporting GPs in managing febrile children are predominantly based on consensus and evidence from hospital emergency care studies, which lack external validation in low-prevalence settings.7,9 The international National Institute for Health and Care Excellence (NICE) guideline for children with feverish illness10,11 proposes a traffic light system, which advises referring a child for specialist consultation if either a ‘red’ or ‘amber (in the absence of a diagnosis and sufficient safety net)’ feature is present. Likewise, the Dutch GP guideline for febrile children12 also bases its referral advice on the presence of single alarm features, all of which are also classified as ‘red’ or ‘amber’ features in the NICE guideline.10 In 2010, a systematic review of mainly hospital emergency care studies identified many of these alarm features as potentially useful in identifying children at high risk of serious infection.13 However, there is still much debate about the diagnostic value of these alarm features in low-prevalence settings.7,8,13
This study aimed to explore to what extent alarm features play a role in referral management of GPs who encounter a febrile child in primary out-of-hours care and to what extent GPs adhere to the national guideline’s advice on referral.
METHOD
Study design
An observational study was performed. Semi-structured, routine clinical practice data were collected of children with fever who had presented to GP out-of-hours care.
Study setting and patient selection
In the Netherlands, patients should in principle first contact the General Practitioner Cooperative (GPC) for out-of-hours primary care. However, within the total amount of out-of-hours demand, 5% of patients will present directly to the emergency department (ED) (that is, self-referral).2 Contacts eligible for this study were children aged <16 years who had a face-to-face consultation with a GP at the GPC and had fever, defined as:
fever reported by parents as the reason for contact;
fever within 24 hours prior to contact; or
a temperature above 38°C measured at the GPC.
How this fits in
The diagnostic value of alarm features of serious infections in low-prevalence settings is unclear. Current guidelines for management of febrile children mainly base their referral advice on the presence of single alarm signs and symptoms. In practice, only 19% of GPs adhered to the guideline’s positive referral advice and 20% of children referred to the emergency department had no alarm features present. It is recommended that further longitudinal follow-up studies investigate the true and false positives/negatives of current referral management and the different reasons why GPs do or do not refer children to the emergency department. This may provide future guidelines with adequate safety-netting advice to fill the gap of insufficient rule-in or rule-out value reached by clinical alarm signs and symptoms alone.
Revisits for the same problem within 7 days of the initial presentation were excluded (Figure 1).
Selection of eligible contacts.
Data collection and data extraction process
The data collection of this study has been described previously.14 In summary, data were collected from all GPC contacts in the Rotterdam-Rijnmond district during March 2008 to February 2009. For the data-extraction process clinical features indicative of a serious infection were derived from the Dutch national GP guideline for febrile children;12 the NICE guideline for feverish illness in children;10 and a systematic review.13 Details on selection of the clinical features were reported previously.11 Selected, closely related features were grouped into 18 alarm signs and symptoms of serious infection (Appendix 1). Whether alarm signs and symptoms were ‘present’, ‘absent’ or ‘not mentioned’ in the patient record were manually recoded using a data-entry computer program Delphi XE (version 15.0). Clinical management by the GP was recoded as ‘referral to ED (yes/no)’.
Statistical analysis
Missing data
Since clinical information was obtained from routine practice data, the study had to manage missing values (Table 1).11 For the purpose of this study, missing values were dealt with in two ways:
Alarm signs and symptoms were assumed to be so relevant that, if present, the GP would document them. Consequently, alarm signs and symptoms ‘not mentioned’ in the patient record were considered ‘absent’ (that is, ill appearance, ABC-instability, unconsciousness, drowsy, inconsolable, cyanosis, shortness of breath, meningeal irritation, neurological signs, vomiting and diarrhoea, dehydration, joint or limb problems, and petechial rash).
For the remaining alarm signs and symptoms missing values were imputed 10 times using the MICE logarithm (R-Project),15 that is, abnormal circulation, signs of urinary tract infection, temperature of ≥40°C, and duration of fever. The imputation model included sex, age, and all alarm signs and symptoms included in the analysis (describing case-mix of the population) and the outcome variable ‘referral to the ED’. Results of the imputation process are displayed in Appendix 2. Vital signs, such as ‘heart frequency’, ‘breathing frequency’, and ‘oxygen saturation’, were reported in only 1% of the patient records and were therefore excluded from the analysis as individual alarming signs.
Characteristics of study population (n = 9794)
Association between alarm features and referral management
The study focused on the Dutch national guideline,12 which advises to refer a febrile child to secondary care if at least one alarm feature is present. Guideline definitions for ‘age <1 month’, ‘abnormal circulation’, ‘meningeal irritation’, ‘petechial rash’, and ‘signs of dehydration’ matched with those of the dataset. For the other guideline features, alarm signs and symptoms were combined or best proxies used in the dataset. The study selected ‘age between 1–3 months’ as a proxy for the guideline feature ‘age between 1–3 months and fever of unknown origin’, ‘ill appearance and/or inconsolable and/or ABC-instability (that is, respiratory or circulatory insufficiency)’ as a proxy for ‘ill appearance’, ‘unconsciousness and/or drowsy’ as a proxy for ‘decreased consciousness’, ‘vomiting and diarrhoea’ as a proxy for ‘persistent vomiting’, and ‘shortness of breath and/or cyanosis’ as a proxy for ‘severe shortness of breath’.
Logistic regression analyses was performed to assess the association between referral to the ED and the presence of alarm features selected from the national guideline. Additionally, the study included alarm features selected from the NICE guideline10 and systematic review,13 that is, ‘neurological signs’, ‘joint or limb problems’, ‘signs of urinary tract infection’, ‘temperature ≥40°C’, and ‘duration of fever’. For the multivariable analyses, the study used multiple imputed data, as much relevant clinical information would be lost by performing a complete case analysis only.
Finally, the study assessed GPs’ adherence to the national guideline by constructing a two-by-two contingency table, that is, referral indication according to guideline versus observed referral to the ED. Statistical analyses were performed with IBM SPSS Software version (20.0).
RESULTS
Characteristics of the study population are displayed in Table 1. In total, 794 (8.1%) of 9794 contacts were followed by a referral to the ED. Frequencies of individual alarm signs and symptoms were generally higher among referred than non-referred children (Table 2). Among the national guideline’s alarm features, ‘age <1 month’, ‘decreased consciousness’, ‘meningeal irritation’, and ‘signs of dehydration’ were most strongly associated with referral. Together, the national guideline-specific alarm features explained 40% of the variability in referral by the GP. Taking into account the alarm features selected from the NICE guideline and the systematic review additionally, the explained variability increased up to maximally 45%.
Association between the presence of alarm features and referral by the GP
Adherence to the national guideline
Table 3 displays guideline adherence by GPs. Overall, 3424 (35%) of 9794 eligible contacts had a positive referral indication, that is, at least one of the guideline-specific alarm features was present. Among these, 633 (19%) of 3424 children were referred to the ED. Among the children with a negative referral indication, that is, none of the guideline-specific alarm features were present, the GP followed the guideline in 6209 (97%) of 6370 contacts. However, within the total group of referred contacts, 161 (20%) of 794 children had no guideline-specific alarm feature present.
GPs’ referral management and guideline adherence12
Table 4 shows the number of alarm features present in children with a positive referral indication. The majority of children for whom the GP overruled the guideline’s advice, that is, decided not to refer the child, had one or two alarm features present. When three or more alarm features were present, nearly all children were referred. Alarm features that GPs predominantly overruled were ‘vomiting’, ‘ill appearance’, ‘abnormal circulation’, and ‘shortness of breath’.
Alarm features among febrile children with a referral indication according to national guidance12
DISCUSSION
Summary
GPs adhered to a positive referral advice by the national guideline in only 19% of the out-of-hours consultations. If only one or two guideline-specific alarm features were present, GPs seemed to be more conservative in referring febrile children to the ED. Alarm features most strongly associated with referral were ‘age <1 month’, ‘decreased consciousness’, ‘meningeal irritation’, and ‘signs of dehydration’, and ‘joint or limb problems’. Even though a negative referral advice by the guideline was adhered to in nearly all of the consultations, 20% of the children referred to the ED had no alarm feature present. This may indicate that for a considerable group of children, GPs base their referral decisions on other reasons than the presence of alarm features.
Strengths and limitations
To the best of the study’s knowledge, this is the first study to provide an insight into the association between guideline and literature-based alarm features and GPs’ referral management in primary out-of-hours care practice.
Similar to the international NICE guideline for febrile children, the Dutch national guideline bases its referral advice on the presence of single alarm features, all of which are classified as ‘red’ or ‘amber’ features in the NICE guideline as well.
For this study a large, multicultural, urban cohort of nearly 10 000 febrile children was used, who presented to primary out-of-hours care. As GPCs function as acute primary care facilities and patients can present on their own initiative, the study believes that this population is likely to be generalisable to other large-scale out-of-hours primary care populations and may be extrapolated to children presenting to paediatric acute assessment units in settings with a low prevalence of serious infections.
As prospective data collection in low-prevalence settings is difficult, the study made use of routine clinical practice data. Consequently, alarm features ‘not mentioned’ in the patient record could either mean ‘not present’ or ‘not looked at by the physician’. It can be assumed that GPs have carefully documented alarm features to either justify their decision to refer a child or to ensure that their reasons for not referring a child were clear. In a consensus meeting, it was decided to use a multiple imputation strategy to limit the amount of clinical information missing and to best approximate true values. A sensitivity analysis on complete cases revealed no major differences in outcomes (data not shown). Therefore, the study assumes the verification bias to be limited.
Comparison with existing literature
Several individual alarm features have been demonstrated to have potential value in identifying (‘ruling-in’) serious infections in children.13 However, their applicability, depends on the setting-specific prevalence of disease. Taking into account the low probability of serious infection in primary care (approximately 1%), the majority of individual alarm features will only raise the posterior probability to about 10% when present.13 As these results were only based on a single primary care study, which lacks external validation, their generalisability to and diagnostic impact in other low-prevalence populations may be questionable.9,16
Both the Dutch GP guideline12 and the international NICE guideline10 base their referral advice on the presence of single alarm features. In the study, it was observed that if one should follow the national guideline, 35% of all children consulted should be referred. Comparable results were reported by others, who validated the Dutch as well as the NICE guideline in low-prevalence17 and intermediate-prevalence populations.17,18 They also found that 16% to 99% of the children consulted received positive referral advice. Consequently, if one were to follow the guidelines’ advice, most children with a serious infection would be referred, yielding high sensitivities (range 81–100%). However, as the prevalence of serious infections in primary care is only about 1%, an enormous group of children would be referred unnecessarily (false positives), resulting in (very) low specificities (range 1–85%). From a safety perspective, this may seem a valid approach; however, the disadvantage may be a considerable overload of children who present at the ED without a serious infection. Besides, such unnecessary referrals may cause harm to children with minor illness through cross-infection with more serious conditions, as well as distress to children and their families.
Interestingly, in clinical practice, the study observed that GPs decided to refer only 19% of the patients with a positive referral indication, of whom the majority had three or more alarm features present. ‘Meningeal irritation’ and ‘decreased consciousness’ were nearly never neglected as alarm signs, whereas ‘ill appearance’ and ‘abnormal circulation’ were quite often overruled. This may suggest that some features have a broader clinical range in primary care than in high-prevalence settings, where these signs and symptoms were identified as important indicators of serious infection.13 From these results, it seems that GPs already apply a certain threshold above which they feel their referral is grounded, that is, they balance the risk between false positive and false negative outcomes. They also seem to share the opinion that combinations of alarm features may do better in ruling in serious infections than single features alone. In line with this finding, others have recently reported on the diagnostic value of three or more ‘red features’ of the NICE traffic light system (E Kerkhof, personal communication, 2013). Unfortunately, the posterior probability of disease was still unsatisfactorily raised to a maximum of about 10% in low-prevalence settings specifically.
Should we then better shift our focus towards ruling out serious infections in low-prevalence settings? Previous reports have indicated that individual alarm features have insufficient rule-out value on their own.6,7,13 However, combinations of absent alarm features may significantly decrease the probability of disease.13 For the majority of children without alarm features present, the GPs in the study seemed quite confident about the absence of a serious infection. However, the difficulty lies in determining the threshold of exactly how many alarm features must be absent to sufficiently rule out serious febrile illness. Clinical prediction rules may, alongside guidelines, help physicians to identify children at low risk of disease.19–25 The only clinical prediction rule developed for primary care specifically showed a promising high sensitivity and low negative likelihood ratio at derivation;6 however, it lacked generalisability on external validation in other low-prevalence populations.17 In addition, another study has shown that other clinical prediction rules developed for hospital emergency care were of limited use in the primary out-of-hours care setting as well.
Finally, another study demonstrated that 20% of the referred children had no alarm feature present. This suggests that other reasons seem important in GPs’ referral decisions.
Implications for research and practice
Even though the exact harms and benefits of currently used clinical guidelines should be further elucidated, the question arises whether it is possible to develop a guideline with only clinical features that sufficiently rule in or rule out serious infections in children consulting in primary care. Future studies may answer this question by exploring the alternative reasons why GPs refer a febrile child; the potentially additive value of inflammatory marker point-of-care tests, such as C-reactive protein to guidelines or clinical prediction rules, as these have shown promising results in adult primary care studies as well as studies performed at paediatric EDs;26–28 and the disease course over time in longitudinal follow-up studies, to provide future guidelines with adequate safety-netting advice to fill the gap of insufficient rule-in or rule-out value reached by clinical alarm features alone.
Acknowledgments
We would like to thank M. de Wilde for his support in data processing and data management. We would like to thank T. Krecinic, Z. Gocmen, M. Hofhuis and M. Rotsteeg for their contrinbution to data management.
Appendix 1. Grouping of alarm features for serious infection
| Grouped alarm features (as coded in the GPC-database) | Total selection of alarm features |
|---|---|
| Parental concern | Parental concern |
| Ill appearance | Clinician’s instinct something is wrong Clinically ill appearance |
| ABC-instability | Respiratory or circulatory insufficiency |
| Unconsciousness | Unconsciousness |
| Drowsy | Child is drowsy Somnolence Reactivity/functional status (decreased) Hypotonia |
| Inconsolable | Child is inconsolable Irritability Changed crying pattern Child is moaning |
| Abnormal circulation | Abnormal skin colour (pale, mottled, ashen) Capillary refill time >2 sec Tachycardia (APLS) |
| Cyanosis | Cyanosis Oxygen saturation <95% |
| Shortness of breath | Shortness of breath Nasal flaring Rapid breathing Changed breathing pattern |
| Meningeal irritation | Neck stiffness Bulging fontanelle |
| Neurological signs | Focal neurological signs Paresis/paralysis Seizures/fits |
| Vomiting & diarrhoea | Vomiting (>2x in disease period) Diarrhoea (>2x in disease period) |
| Dehydration | Dry mucous membranes Sunken eyes Decreased skin elasticity Reduced urine output Hypotension (APLS) Poor feeding |
| Joint or limb problems | Swelling of limb or joint Non-weight bearing limb Not using an extremity |
| Signs of urinary tract infection | Urinary frequency Dysuria Tummy ache (without other focus for fever) |
| Petechial rash | Petechial rash Purpura |
| Temperature ≥40°C | Measured at home or at GPC |
| Duration of fever | Duration of fever (>38.0°C) in days |
Appendix 2. Results of the multiple imputation process
| Alarm features | Present n (%) | Absent n (%) |
|---|---|---|
| Temperature at GPC in °C, mean (SE) | 38.4 (0.02) | |
| Abnormal circulation | 636 (6.5) | 9158 (93.5) |
| Signs of UTI | 1213 (12.4) | 8581 (87.6) |
| Temperature ≥40°C | 2811 (28.7) | 6983 (71.3) |
| Duration of fever: | ||
| Started today | 2560 (26.1) | |
| 1 day | 2199 (22.5) | |
| 2 days | 1543 (15.8) | |
| 3 days | 1669 (17.0) | |
| 4 days | 885 9.0) | |
| 5 days | 451 (4.6) | |
| 6 days | 154 1.6) | |
| ≥7 days | 333 3.4) |
Missing values were imputed 10 times with MICE (R-project) for the alarm features ‘Temperature at GPC’, ‘Abnormal circulation’, ‘Signs of UTI’, ‘Temperature ≥40°C’, and ‘Duration of fever’. All other alarming signs reported had no missing data and frequencies are displayed in Table 1.
Notes
Funding
This study was funded by an unrestricted grant from European Container Terminals BV, Rotterdam, The Netherlands.
Ethical approval
The institution’s medical ethics committee reviewed the study and the requirement for informed consent was waived (MEC-2012-378).
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors have declared no competing interests.
Discuss this article
Contribute and read comments about this article: www.bjgp.org/letters
- Received August 1, 2013.
- Revision received October 16, 2013.
- Accepted November 8, 2013.
- © British Journal of General Practice 2014
REFERENCES
In this issue
Jump to section
Keywords
More in this TOC Section
Related Articles
Cited By...
