Summary
GPs adhered to a positive referral advice by the national guideline in only 19% of the out-of-hours consultations. If only one or two guideline-specific alarm features were present, GPs seemed to be more conservative in referring febrile children to the ED. Alarm features most strongly associated with referral were ‘age <1 month’, ‘decreased consciousness’, ‘meningeal irritation’, and ‘signs of dehydration’, and ‘joint or limb problems’. Even though a negative referral advice by the guideline was adhered to in nearly all of the consultations, 20% of the children referred to the ED had no alarm feature present. This may indicate that for a considerable group of children, GPs base their referral decisions on other reasons than the presence of alarm features.
Strengths and limitations
To the best of the study’s knowledge, this is the first study to provide an insight into the association between guideline and literature-based alarm features and GPs’ referral management in primary out-of-hours care practice.
Similar to the international NICE guideline for febrile children, the Dutch national guideline bases its referral advice on the presence of single alarm features, all of which are classified as ‘red’ or ‘amber’ features in the NICE guideline as well.
For this study a large, multicultural, urban cohort of nearly 10 000 febrile children was used, who presented to primary out-of-hours care. As GPCs function as acute primary care facilities and patients can present on their own initiative, the study believes that this population is likely to be generalisable to other large-scale out-of-hours primary care populations and may be extrapolated to children presenting to paediatric acute assessment units in settings with a low prevalence of serious infections.
As prospective data collection in low-prevalence settings is difficult, the study made use of routine clinical practice data. Consequently, alarm features ‘not mentioned’ in the patient record could either mean ‘not present’ or ‘not looked at by the physician’. It can be assumed that GPs have carefully documented alarm features to either justify their decision to refer a child or to ensure that their reasons for not referring a child were clear. In a consensus meeting, it was decided to use a multiple imputation strategy to limit the amount of clinical information missing and to best approximate true values. A sensitivity analysis on complete cases revealed no major differences in outcomes (data not shown). Therefore, the study assumes the verification bias to be limited.
Comparison with existing literature
Several individual alarm features have been demonstrated to have potential value in identifying (‘ruling-in’) serious infections in children.13 However, their applicability, depends on the setting-specific prevalence of disease. Taking into account the low probability of serious infection in primary care (approximately 1%), the majority of individual alarm features will only raise the posterior probability to about 10% when present.13 As these results were only based on a single primary care study, which lacks external validation, their generalisability to and diagnostic impact in other low-prevalence populations may be questionable.9,16
Both the Dutch GP guideline12 and the international NICE guideline10 base their referral advice on the presence of single alarm features. In the study, it was observed that if one should follow the national guideline, 35% of all children consulted should be referred. Comparable results were reported by others, who validated the Dutch as well as the NICE guideline in low-prevalence17 and intermediate-prevalence populations.17,18 They also found that 16% to 99% of the children consulted received positive referral advice. Consequently, if one were to follow the guidelines’ advice, most children with a serious infection would be referred, yielding high sensitivities (range 81–100%). However, as the prevalence of serious infections in primary care is only about 1%, an enormous group of children would be referred unnecessarily (false positives), resulting in (very) low specificities (range 1–85%). From a safety perspective, this may seem a valid approach; however, the disadvantage may be a considerable overload of children who present at the ED without a serious infection. Besides, such unnecessary referrals may cause harm to children with minor illness through cross-infection with more serious conditions, as well as distress to children and their families.
Interestingly, in clinical practice, the study observed that GPs decided to refer only 19% of the patients with a positive referral indication, of whom the majority had three or more alarm features present. ‘Meningeal irritation’ and ‘decreased consciousness’ were nearly never neglected as alarm signs, whereas ‘ill appearance’ and ‘abnormal circulation’ were quite often overruled. This may suggest that some features have a broader clinical range in primary care than in high-prevalence settings, where these signs and symptoms were identified as important indicators of serious infection.13 From these results, it seems that GPs already apply a certain threshold above which they feel their referral is grounded, that is, they balance the risk between false positive and false negative outcomes. They also seem to share the opinion that combinations of alarm features may do better in ruling in serious infections than single features alone. In line with this finding, others have recently reported on the diagnostic value of three or more ‘red features’ of the NICE traffic light system (E Kerkhof, personal communication, 2013). Unfortunately, the posterior probability of disease was still unsatisfactorily raised to a maximum of about 10% in low-prevalence settings specifically.
Should we then better shift our focus towards ruling out serious infections in low-prevalence settings? Previous reports have indicated that individual alarm features have insufficient rule-out value on their own.6,7,13 However, combinations of absent alarm features may significantly decrease the probability of disease.13 For the majority of children without alarm features present, the GPs in the study seemed quite confident about the absence of a serious infection. However, the difficulty lies in determining the threshold of exactly how many alarm features must be absent to sufficiently rule out serious febrile illness. Clinical prediction rules may, alongside guidelines, help physicians to identify children at low risk of disease.19–25 The only clinical prediction rule developed for primary care specifically showed a promising high sensitivity and low negative likelihood ratio at derivation;6 however, it lacked generalisability on external validation in other low-prevalence populations.17 In addition, another study has shown that other clinical prediction rules developed for hospital emergency care were of limited use in the primary out-of-hours care setting as well.
Finally, another study demonstrated that 20% of the referred children had no alarm feature present. This suggests that other reasons seem important in GPs’ referral decisions.
