Translation of clinical prediction rules for febrile children to primary care practice: an observational cohort study

Yvette van Ierland; Gijs Elshout; Marjolein Y Berger; Yvonne Vergouwe; Marcel de Wilde; Johan van der Lei; Henriëtte A Mol; Rianne Oostenbrink

doi:10.3399/bjgp15X684373

Abstract

Background Clinical prediction rules (CPRs) to identify children with serious infections lack validation in low-prevalence populations, which hampers their implementation in primary care practice.

Aim To evaluate the diagnostic value of published CPRs for febrile children in primary care.

Design and setting Observational cohort study among febrile children (<16 years) who consulted five GP cooperatives (GPCs) in the Netherlands.

Method Alarm signs of serious infection and clinical management were extracted from routine clinical practice data and manually recoded with a structured electronic data-entry program. Eight CPRs were selected from literature. CPR-variables were matched with alarm signs and CPRs were applied to the GPC-population. ‘Referral to emergency department (ED)’ was used as a proxy outcome measure for ‘serious infection’. CPR performance was assessed by calibration analyses, sensitivity, specificity, and area under the ROC-curve (ROC-area).

Results A total of 9794 GPC-contacts were eligible, 54% male, median age 2.3 years (interquartile range 1.0–4.6 years) and 8.1% referred to ED. Frequencies of CPR-variables varied from 0.5% (cyanosis, drowsy) to 25% (temperature ≥40°C). Alarm signs frequently included in CPRs were ‘ill appearance’, ‘inconsolable’, and ‘abnormal circulatory or respiratory signs’. The height of the CPR’s predicted risks generally corresponded with being (or not being) referred to the ED in practice. However, calibration-slopes indicated that three CPRs underestimated the risk of serious infection in the GPC-population. Sensitivities ranged from 42% to 54%, specificities from 68% to 89%. ROC-areas ranged from 0.52 to 0.81, with best performance of CPRs for children aged <3 months.

Conclusion Published CPRs performed moderately well in the primary out-of-hours care population. Advice is given on how to improve translation of CPRs to primary care practice.

INTRODUCTION

Most children who present to primary (out-of-hours) care have fever as one of their main complaints.¹,² Febrile children are at risk of serious infections, such as meningitis or pneumonia, which are important causes of morbidity and mortality.³–⁵ However, the prevalence of serious infections in primary care is low⁶ and physicians have the challenging task of distinguishing children at high risk of serious infections from those with self-limiting disease. Only a few studies on the identification of serious infections in primary care have been published so far.⁶,⁷ Consequently, practice guidelines are mainly based on consensus of expert opinion and scientific evidence collected from secondary and tertiary emergency care studies.⁸,⁹ To complement practice guidelines, clinical prediction rules (CPRs) could be powerful tools to improve clinical decision making on the basis of combinations of clinical signs and symptoms.¹⁰ However, most published CPRs for serious infections have been developed predominantly at hospital emergency departments (EDs),¹¹–¹³ with the lack of external validation in low-prevalence populations hampering their implementation in primary care practice.¹¹,¹³,¹⁴ The present study aimed to assess the applicability and diagnostic value of published CPRs for serious infections in febrile children consulting primary out-of-hours care.

METHOD

Study design

As part of an observational study, semi-structured, routine clinical practice data were prospectively collected on children who had presented to out-of-hours primary care (workdays from 5 pm to 8 am, and the entire weekend) with fever. The diagnostic value of published CPRs for serious infections was assessed, defining ‘referral to ED’ as the outcome measure.

Study setting and selection of patients

The out-of-hours healthcare system in the Netherlands and data collection of this study have been published previously.¹⁵ In summary, all contacts of children <16 years that had taken place at five GP cooperatives (GPCs) of the Rotterdam Rijnmond-district (collaboration of >250 GP-practices) between March 2008 and February 2009 were selected. Eligible contacts were those concerning children who had a face-to-face consultation with the GP and reported fever as the reason for contact, had fever within the 24 hours before contact, or had a (rectal or tympanic) temperature >38°C measured at the GPC. Re-contacts for the same problem within 7 days of the initial presentation were excluded from the main analyses.

How this fits in

Published clinical prediction rules (CPRs) to identify children with serious infections lack validation in low-prevalence populations, which hampers their implementation in primary care practice. The present study shows that published CPRs perform only moderately well in the primary out-of-hours care population with limited rule-out value. Improved translation of CPRs to primary out-of-hours care could be achieved by the introduction of structured documentation of vital signs, application of inflammatory marker point-of-care tests, introduction of follow-up contacts, and the identification of predictors with diagnostic value in the primary care setting specifically.

Extraction of relevant clinical signs

Clinical features indicative of serious infections were derived from one systematic review¹² and two published guidelines on management of febrile children.⁸,¹⁶ Features were included that:

had a high predictive value (positive likelihood ratio >5.0 or negative likelihood ratio <0.2);
were mentioned in at least two of the three data sources;
did not represent a diagnosis; and
were not prone to high inter-observer variability (for example, auscultatory sounds).¹⁷

The selected features were grouped into 18 ‘alarm signs’ (Appendix 1). For eligible contacts, it was manually recoded whether alarm signs were ‘present’, ‘absent’, or ‘not mentioned’ in the patient record, using the data-entry computer program Embarcadero Delphi XE (version 15.0). Clinical management by the GP was recoded as ‘referral to ED’, ‘follow-up appointment at GP(C)’, or ‘no follow-up’.

Selection of CPRs and translation to the primary out-of-hours care population

Eligible CPRs were selected on the basis of two important reviews on this topic,¹¹,¹² and an additional literature search of published CPRs for serious infections in children (published between 1975 and 2012, and relevant to developed countries; Appendix 2). Selected CPRs were deemed to:

have clinical signs and symptoms as predictors;
have no more than one laboratory test as a predictor variable, because these are unavailable at the GPC;
have a composite outcome of serious infections; and
advise on management strategies or give a risk score.

Eight of 32 CPRs extracted from literature were included in the final analyses (Table 1).⁶,¹⁸–²³ Variables of the selected CPRs were matched with the alarm signs in the GPC-dataset. In case alarm signs were not entirely identical to the original CPR variables, best proxy variables were used. When CPR variables were missing in the GPC-dataset, it was assumed these were absent.

View this table:

Table 1.

Overview of selected clinical prediction rules

The unavailability of data on exact diagnoses hampered the verification of outcome diagnoses in the routine clinical practice dataset. As in primary care, identification of febrile children at risk of a serious infection (that is, requiring specialist assessment) is often more important than confirming the exact diagnosis,¹⁴,²⁴ ‘referral to ED’ was used as a proxy for ‘serious infection’. This proxy was validated among a subset of GP-referred febrile children who presented to the ED of the nearby Sophia Children’s Hospital during the out-of-hours period (January 2006 to July 2009; N = 376).²⁵ It was observed that 66% of these GP-referred children required some form of extensive diagnostic interventions (such as, blood culture or lumbar puncture), extensive therapeutic interventions (such as, intravenous [IV]-medication or aerosol treatment), or hospitalisation, indicating the presence of a serious febrile illness. Only 395 (4%) of 9794 GPC-contacts had a second contact for the same complaint within 7 days, of which 67 (0.7%) were referred to the ED. Figures were comparable for children who had or had not been prescribed antibiotics at first consultation.

Missing data

As clinical information was obtained from routine practice data, there were some missing values. A consensus-meeting with one GP, two paediatricians, and two residents (general practice and paediatrics) decided, for the purpose of this study, to deal with missing values in two way. As GPs are taught to recognise alarm signs in febrile children, it was assumed that alarm signs were always documented when present and, consequently, when alarm signs were ‘not mentioned’ in the patient record, these were considered to be ‘absent’. Also for continuous variables (such as, temperature and duration of fever), missing values were replaced by mean values.

Statistical analyses

Patient characteristics and frequencies of alarm signs were analysed using descriptive statistics. For some CPRs based on multivariable logistic regression models, two separate but closely related variables were combined into one alarm sign (such as ‘tachypnoea’ and ‘chest wall retractions’ into ‘shortness of breath’) or categorical variables (such as mild, moderate, or severe) were dichotomised into ‘present’ or ‘absent’ for application in the GPC-population. For such CPR variables the β-coefficients were recalculated as a weighted mean on the basis of the original β-coefficients and the number of patients with the variable present at derivation.

Selected CPRs were applied to the eligible GPC-population, within the age ranges for which the rules were originally derived. For CPRs which predict a high or low risk of serious infection (CPRs 1–3), calibration was assessed by calculating the percentage of referral in the predicted high- and low-risk groups. For CPRs which gave a continuous risk prediction (CPRs 4–8), first the linear predictor was calculated, which is the sum product of regression coefficients of the rule and the variable values (lp = α + β₁·x₁ + β₂·x₂ + … + β_i·x_i, in which α is the intercept and β₁β_i are the regression coefficients of the variables x₁x_i). Calibration was assessed by calculating the observed frequency of referral among patient groups based on percentiles of the predicted risk (that is, lp-outcome) and calculation of calibration slopes. Discriminative ability was assessed by calculating sensitivity, specificity, and likelihood ratios for CPRs 1–3 and areas under the receiver operating characteristic curve (ROC-area) for CPRs 4–8. Statistical analyses were performed with SPSS PASW software (version 17.0.2).

RESULTS

Description of the GPC-population

In total, 9794 GPC-contacts were eligible for analyses (Figure 1). General patient characteristics and clinical features are shown in Table 2. Vital signs such as heart rate, respiratory rate, and oxygen saturation were documented in only 2% of the patients (data not shown). Referral to the ED was required for 794 (8.1%) contacts.

Figure 1.

Selection of eligible contacts.

View this table:

Table 2.

Characteristics of study population (n = 9794)

Performance of CPRs applied to the GPC-population

Eight CPRs were applicable to the primary out-of-hours care setting (Table 1). Observed frequencies of CPR variables were generally low in the GPC-population and ranged from 0.5% (cyanosis/drowsy) to 25% (temperature ≥40°C; Table 3). For CPRs 1–3, observed referral frequencies varied from 4% to 38% among the patients predicted as ‘low-risk’ and from 13% to 78% among those predicted as ‘high-risk’ (Table 4). For CPRs 4–8, the distribution of patients over the percentile groups of the predicted risk differed considerably per CPR, with the lowest percentile groups accounting for minimally 6% to maximally 44% of the total population (Table 5). The observed percentage of children referred to the ED was generally low for those in the lowest percentile groups (range 0–20%). Observed referral frequencies of children in the upper percentile groups ranged from 17% (CPR 5) to 100% (CPR 7). Calibration slopes varied from 0.17 (CPR 6) to 2.05 (CPR 8), with three rules having a slope >1, which indicates that the effects of CPR variables were larger in the GPC- population than in the derivation-population (Appendix 3). Sensitivities of CPRs 1–3 ranged from 42% to 54%, and were lower than those reported in derivation settings. In contrast, specificities of CPRs 2 and 3 were higher (>86% versus 39% and 35% at derivation), as were positive and negative likelihood ratios (Table 4). Discriminative abilities of CPRs 4–8 varied widely, but were to some extent comparable with ROC-areas reported in the derivation studies (Table 5). CPR 6 had the lowest ROC-area of 0.52, whereas the two rules developed for young children showed the best discriminative abilities with ROC-areas of 0.77 (CPR 4) and 0.81 (CPR 7).

View this table:

Table 3.

Model predictors used and frequencies in the GPC-population

View this table:

Table 4.

Performance of clinical prediction rules with a high/low risk prediction (CPRs 1–3)

View this table:

Table 5.

Performance of clinical prediction rules with continuous risk prediction (CPRs 4–8)

DISCUSSION

Summary

The present study demonstrated that published CPRs for serious infections, mainly derived at hospital EDs, performed only moderately well in the primary out-of-hours care setting using ‘referral to ED’ as the outcome measure. Most CPR variables were observed to be reported positively in a low frequency in the GPC-population. Limited rule-out value was found for CPRs that classified children into high- or low-risk groups. Use of CPRs which gave a continuous risk prediction was too moderate to be directly applicable to clinical primary care practice.

Strengths and limitations

This study is the first to assess the diagnostic value of several published CPRs for serious infections in an urban, multi-ethnic, out-of-hours primary care cohort of nearly 10 000 contacts of febrile children. It is anticipated that these results will be valuable to many other countries where primary care is provided by GPs or where out-of-hours care has similarly shifted towards large-scale cooperatives.²⁶ As prospective studies on serious infections in low prevalence settings are logistically challenging and time consuming, routine clinical practice data were used in this study. Consequently, the results should be considered in the context of some difficulties and limitations elicited by this ‘second best’ approach.¹⁴,²⁴

First, ‘referral to ED’ was used as a proxy outcome measure for ‘serious infection’. It is acknowledged that using a differently defined, although well-correlated, outcome measure may be suboptimal and may have resulted in an overestimation of the prevalence of serious infections in the study population (for example, reflected by calibration slopes >1). For each of the selected CPRs, however, ‘serious infection’ has been defined differently. As for primary care physicians, identifying children at high risk of serious infections is more important than knowing exact diagnoses, ‘referral to ED’ may be an outcome, which captures all of these different definitions of serious infections. This proxy was validated but some bias cannot be excluded, as for some children clinical management may have been based on factors other than the presence of alarm signs only (for example, physician’s experience, demanding/concerned parents, or the need for diagnostic certainty).¹⁵ On the other hand, some children may have been adequately treated with antibiotics rather than being referred by the GP.

Secondly, there was a considerable number of missing (‘not mentioned’) values. As assumptions on the mechanism of ‘missingness’ may be diverse, missing values were replaced on the basis of clinical rationale for the purpose of this study. To evaluate potential bias arising from this approach, sensitivity analyses were performed with missing values imputed on the basis of correlations between missing values and available information of other variables.²⁷ For CPRs 4–8, these secondary analyses showed similar ROC-areas but calibration slopes closer to 1 (that is, better model fit). For CPRs 2 and 3, particular higher sensitivities were observed (73% and 62%, respectively) and lower negative likelihood ratios (0.4 and 0.6, respectively), which indicates that the main analyses may ‘err’ on the safe side by underestimating rather than overestimating the CPR’s performances.

Comparison with existing literature

The present finding that most CPRs performed only moderately well in the low prevalence setting may, next to the methodological limitations of this study, be explained by the specific characteristics of primary care practice itself. GPs constantly have to balance the risk of missing a serious infection versus unnecessary referral.⁷ This difficulty particularly accounts for children with an unclear presentation (‘grey area’),¹⁴ which was previously demonstrated in studies among children hospitalised for meningococcal disease²⁸–³¹ and malpractice lawsuits.³²,³³ Clinical decision support by CPRs may be helpful in this diagnostic dilemma; however, in the low prevalence setting, high rule-out value should be achieved to reduce the number of false negative patients.⁷,²⁴ Unfortunately, in this study it was shown that the published CPRs which predicted a high or low risk of serious infection showed insufficient rule-out value (low sensitivities and high negative likelihood ratios). Most CPRs that gave a continuous risk prediction could not discriminate well between the middle percentile-groups of the predicted risk, and observed referral frequencies varied considerably (Appendix 3).

Possible explanations for this discrepancy in performance of CPRs at the GPC and ED, may be that in the primary care setting observed frequencies of CPR variables are low, vital signs are barely measured,³⁴ and additional diagnostic tests, such as inflammatory markers, are unavailable. These issues reduced the heterogeneity of CPR variable outcomes, and thus of the predicted risks for each patient in the GPC population (that is, less spread of the predicted risks). Unavailability of predictor variables in the GPC setting may have negatively influenced the performance of some CPRs more than others. Also, some predictor variables are likely to be better predictors of serious infection in the ED setting than the GPC setting.

Unfortunately, the only CPR derived in a primary care setting itself⁶ showed no diagnostic value in the present GPC-population. Previously, others had also demonstrated only marginal rule-out value for this CPR,¹³,³⁵ which further underscores the importance of external validation before implementation in clinical practice.¹⁰,¹⁴

Implications for research and practice

How do the present results support the translation of existing CPRs to primary care settings? Given the importance of vital signs in most CPRs, structured vital sign measurements at GPCs would be advised. The present study suggests that published CPRs should be updated with variables available in and relevant to the primary care setting.³⁶ In this way, CPRs may better discriminate between seriously ill children and those in the ‘grey area’ who have a less clear clinical presentation. The strong diagnostic value of inflammatory markers, as already demonstrated in adult primary care,³⁷–³⁹ may favour their implementation in primary care practice to further improve diagnostic discrimination. However, it should be noted that no inflammatory marker has perfect discriminative ability on its own and clinical observation remains essential (as is reflected by CPRs combining clinical variables with inflammatory markers). Finally, a follow-up period after the initial GPC-contact could contribute to differentiation between evolving serious infections from self-limiting viral disease.

In light of the methodological difficulties, the present study shows that published CPRs performed moderately well in the primary out-of-hours care population with limited rule-out value. Most CPR variables were observed to be reported positively in a low frequency in the GPC-population. Improved translation of CPRs to primary out-of-hours care could be achieved by introduction of structured documentation of vital signs, application of inflammatory marker point-of-care tests, introduction of follow-up contacts, and identification of predictors with diagnostic value in the primary care setting specifically.

Acknowledgments

We acknowledge EJ van Dijk for providing the data collected at the GPC and thank T Krecinic, Z Gocmen, M Hofhuis, and M Rotsteeg for their contribution to the data management of this study.

Appendix 1. Grouping of alarm signs for serious infection

Grouped alarm signs (as coded in the GPC-database)	Total selection of alarm signs
Parental concern	Parental concern
Ill appearance	Clinician’s instinct something is wrong Clinically ill appearance
ABC-instability	ABC-instability
Unconsciousness	Unconsciousness
Drowsy	Child is drowsy Somnolence Reactivity/functional status (decreased) Hypotonia
Inconsolable	Child is inconsolable Irritability Changed crying pattern Child is moaning
Abnormal circulation	Abnormal skin colour (pale, mottled, ashen) Capillary refill time >2 seconds Tachycardia
Cyanosis	Cyanosis Oxygen saturation <95%
Shortness of breath	Shortness of breath Nasal flaring Rapid breathing Changed breathing pattern
Meningeal irritation	Meningeal irritation Neck stiffness Bulging fontanelle
Neurological signs	Focal neurological signs Paresis/paralysis Seizures/fits
Vomiting and diarrhoea	Vomiting (>2 times in disease period) Diarrhoea (>2 times in disease period)
Dehydration	Dry mucous membranes Sunken eyes Decreased skin elasticity Reduced urine output Hypotension (APLS) Poor feeding
Extremity problems	Swelling of limb or joint Non-weight bearing limb Not using an extremity
Signs of urinary tract infection	Urinary frequency Dysuria Tummy ache (without other focus for fever)
Petechial rash	Petechial rash Purpura
Temperature ≥40°C	Measured at home or at GP cooperative
Duration of fever	Duration of fever (>38.0°C) in days

ABC = airway, breathing, circulation

Appendix 2. Electronic search strategy for clinical prediction rules

Pubmed
(Decision Trees[mesh] OR decision rule*[tw] OR decision tree*[tw] OR prediction rule*[tw] OR predictive rule*[tw] OR decision model*[tw] OR prediction model*[tw] OR predictive model*[tw] OR decision analysis model*[tw] OR risk score*[tw]) AND (child[mesh] OR child[tiab] OR children[tiab] OR pediatric*[tw] OR infant*[tw]) AND (“Arthritis, Infectious”[Mesh] OR “Bone Diseases, Infectious”[Mesh] OR “Community-Acquired Infections”[Mesh] OR “Respiratory Tract Infections”[Mesh] OR “Sepsis”[Mesh] OR “Skin Diseases, Infectious”[Mesh] OR “Soft Tissue Infections”[Mesh] OR “Urinary Tract Infections”[Mesh] OR “Meningitis”[Mesh] OR meningitis[tw] OR serious infection*[tw] OR serious bacterial infection*[tw] OR severe bacterial infection*[tw] OR severe infection*[tw] OR “Gastroenteritis”[Mesh])

Embase
(‘Decision Tree’/de OR ((decision* OR predict* OR risk*) NEAR/3 (rule* OR model* OR algorithm* OR aid OR score* OR tree*)):de,ab,ti) AND (child/exp OR (child* OR pediatric* OR infant*):de,ab,ti) AND (‘infectious arthritis’/exp OR ‘hematogenous osteomyelitis’/exp OR ‘communicable disease’/exp OR ‘respiratory tract infection’/exp OR ‘sepsis’/exp OR ‘skin infection’/exp OR ‘soft tissue infection’/exp OR ‘urinary tract infection’/exp OR ‘meningitis’/exp OR ‘gastroenteritis’/exp OR (serious* NEAR/3 infection*):de,ab,ti)

Appendix 3. Distribution of contacts over the percentiles of the predicted risk and frequency of referral within groups

Percentiles of the predicted risk	CPR4 *Pantell et al* ¹⁹**		CPR 5 *Bleeker et al* ²⁰**		CPR 6 *Berger et al* ²¹**		CPR 7 YICSSG ²²		CPR 8 *Brent et al* ²³**

	n	Referral to ED, %	n	Referral to ED, %	n	Referral to ED, %	n	Referral to ED, %	n	Referral to ED, %
0–10^th	–	–	893	6	–	–	7	0	610	6
10–20^th	65	20	228	8	–	–	–	–	–	–
20–30^th	–	–	744	5	1052	12	–	–	–	–
30–40^th	16	75	281	7	–	–	65	40	–	–
40–50^th	6	67	554	4	29	45	–	–	6926	4
50–60^th	48	25	811	9	322	11	–	–	–	–
60–70^th	7	86	566	4	–	–	–	–	–	–
70–80^th	20	65	464	13	685	10	20	100	291	34
80–90^th	24	92	705	9	35	46	10	80	1657	11
90–100^th	18	94	563	26	259	17	12	100	278	67
Total	204	49	5809	9	2382	13	114	58	9762	8

CPR = clinical prediction rule. ED = emergency department. YICSSG = Young Infants Clinical Signs Study Group.

Notes

Funding

This work was supported by an unrestricted grant from Europe Container Terminals B.V. The funder had no role in study design, the collection, analyses or interpretation of data, writing the report or the decision to submit the manuscript for publication. Rianne Oostenbrink is supported by a personal grant of the European Society for Paediatric Infectious Diseases (ESPID Fellowship 2009).

Ethical approval

The institution’s medical ethics committee reviewed the study and the requirement for informed consent was waived (MEC-2012–378).

Provenance

Freely submitted; externally peer reviewed.

Competing interests

The authors have declared no competing interests.

Discuss this article

Contribute and read comments about this article: bjgp.org/letters

Received June 2, 2014.
Revision requested June 23, 2014.
Accepted July 28, 2014.

REFERENCES

1.↵
1. Moll van Charante EP,
2. van Steenwijk-Opdam PC,
3. Bindels PJ
(2007) Out-of-hours demand for GP care and emergency services: patients’ choices and referrals by general practitioners and ambulance services. BMC Fam Pract 8:46.
OpenUrl CrossRef PubMed
2.↵
1. Bruijnzeels MA,
2. Foets M,
3. van der Wouden JC,
4. et al.
(1998) Everyday symptoms in childhood: occurrence and general practitioner consultation rates. Br J Gen Pract 48(426):880–884.
OpenUrl Abstract/FREE Full Text
3.↵
1. Bateman SL,
2. Seed PC
(2010) Procession to pediatric bacteremia and sepsis: covert operations and failures in diplomacy. Pediatrics 126(1):137–150.
OpenUrl Abstract/FREE Full Text
4.
1. Prayle A,
2. Atkinson M,
3. Smyth A
(2011) Pneumonia in the developed world. Paediatr Respir Rev 12(1):60–69.
OpenUrl CrossRef PubMed
5.↵
1. Saez-Llorens X,
2. McCracken GH Jr.
(2003) Bacterial meningitis in children. Lancet 361(9375):2139–2148.
OpenUrl CrossRef PubMed
6.↵
1. Van den Bruel A,
2. Aertgeerts B,
3. Bruyninckx R,
4. et al.
(2007) Signs and symptoms for diagnosis of serious infections in children: a prospective study in primary care. Br J Gen Pract 57(540):538–546.
OpenUrl Abstract/FREE Full Text
7.↵
1. Thompson MJ,
2. Harnden A,
3. Del Mar C
(2009) Excluding serious illness in feverish children in primary care: restricted rule-out method for diagnosis. BMJ 338:b1187.
OpenUrl FREE Full Text
8.↵
1. Berger MY,
2. Boomsma LJ,
3. Albeda FW,
4. et al.
(2008) Guideline. Children with fever, [NHG-standaard Kinderen met koorts] http://nhg.artsennet.nl/kenniscentrum/k_richtlijnen/k_nhgstandaarden/Samenvattingskaartje-NHGStandaard/M29_svk.htm (accessed 13 Feb 2014).
9.↵
1. National Institute for Health and Care Excellence
(May, 2013) Feverish illness in children, http://guidance.nice.org.uk/CG160 (accessed 27 Nov 2014).
10.↵
1. Reilly BM,
2. Evans AT
(2006) Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med 144(3):201–209.
OpenUrl CrossRef PubMed
11.↵
1. Maguire JL,
2. Kulik DM,
3. Laupacis A,
4. et al.
(2011) Clinical prediction rules for children: a systematic review. Pediatrics 128(3):e666–e677.
OpenUrl Abstract/FREE Full Text
12.↵
1. Van den Bruel A,
2. Haj-Hassan T,
3. Thompson M,
4. et al.
(2010) Diagnostic value of clinical features at presentation to identify serious infection in children in developed countries: a systematic review. Lancet 375(9717):834–845.
OpenUrl CrossRef PubMed
13.↵
1. Thompson M,
2. Van den Bruel A,
3. Verbakel J,
4. et al.
(2012) Systematic review and validation of prediction rules for identifying children with serious infections in emergency departments and urgent-access primary care. Health Technol Assess 16(15):1–100.
OpenUrl CrossRef PubMed
14.↵
1. Oostenbrink R,
2. Thompson M,
3. Steyerberg EW,
4. ERNIE members
(2012) Barriers to translating diagnostic research in febrile children to clinical practice: a systematic review. Arch Dis Child 97(7):667–672.
OpenUrl Abstract/FREE Full Text
15.↵
1. Elshout G,
2. van Ierland Y,
3. Bohnen AM,
4. et al.
(2013) Alarm signs and antibiotic prescription in febrile children in primary care: an observational cohort study. Br J Gen Pract doi:10.3399/bjgp13X669158.
OpenUrl Abstract/FREE Full Text
16.↵
1. National Institute for Health and Care Excellence
(May, 2007) Feverish illness in children — Assessment and initial management in children younger than 5 years, http://www.nice.org.uk/CG047 (accessed 27 Nov 2014).
17.↵
1. Muris JWM
(1990) Lung auscultation in general practice. A literature survey. [in Dutch]. Huisarts Wet 33(7):258–262.
OpenUrl
18.↵
1. Thompson M,
2. Coad N,
3. Harnden A,
4. et al.
(2009) How well do vital signs identify children with serious infections in paediatric emergency care? Arch Dis Child 94(11):888–893.
OpenUrl Abstract/FREE Full Text
19.↵
1. Pantell RH,
2. Newman TB,
3. Bernzweig J,
4. et al.
(2004) Management and outcomes of care of fever in early infancy. JAMA 291(10):1203–1212.
OpenUrl CrossRef PubMed
20.↵
1. Bleeker SE,
2. Derksen-Lubsen G,
3. Grobbee DE,
4. et al.
(2007) Validating and updating a prediction rule for serious bacterial infection in patients with fever without source. Acta Paediatr 96(1):100–104.
OpenUrl PubMed
21.↵
1. Berger RM,
2. Berger MY,
3. van Steensel-Moll HA,
4. et al.
(1996) A predictive model to estimate the risk of serious bacterial infections in febrile infants. Eur J Pediatr 155(6):468–473.
OpenUrl CrossRef PubMed
22.↵
1. Young Infants Clinical Signs Study Group (YICSSG)
(2008) Clinical signs that predict severe illness in children under age 2 months: a multicentre study. Lancet 371(9607):135–142.
OpenUrl CrossRef PubMed
23.↵
1. Brent AJ,
2. Lakhanpaul M,
3. Thompson M,
4. et al.
(2011) Risk score to stratify children with suspected serious bacterial infection: observational cohort study. Arch Dis Child 96(4):361–367.
OpenUrl Abstract/FREE Full Text
24.↵
1. Buntinx F,
2. Mant D,
3. Van den Bruel A,
4. et al.
(2011) Dealing with low-incidence serious diseases in general practice. Br J Gen Pract doi:10.3399/bjgp11X548974.
OpenUrl Abstract/FREE Full Text
25.↵
1. van Ierland Y,
2. van Veen M,
3. Huibers L,
4. et al.
(2011) Validity of telephone and physical triage in emergency care: the Netherlands Triage System. Fam Pract 28(3):334–341.
OpenUrl
26.↵
1. Thomson S,
2. Osborn R,
3. Squires D,
4. Reed SJ
, eds (2011) International profiles of health care systems (The Commonwealth Fund, New York, NY).
27.↵
1. Donders AR,
2. van der Heijden GJ,
3. Stijnen T,
4. Moons KG
(2006) Review: a gentle introduction to imputation of missing values. J Clin Epidemiol 59(10):1087–1091.
OpenUrl CrossRef PubMed
28.↵
1. Thompson MJ,
2. Ninis N,
3. Perera R,
4. et al.
(2006) Clinical recognition of meningococcal disease in children and adolescents. Lancet 367(9508):397–403.
OpenUrl CrossRef PubMed
29.
1. Riordan FA,
2. Thomson AP,
3. Sills JA,
4. Hart CA
(1996) Who spots the spots? Diagnosis and treatment of early meningococcal disease in children. BMJ 313(7067):1255–1256.
OpenUrl FREE Full Text
30.
1. Sorensen HT,
2. Moller-Petersen J,
3. Krarup HB,
4. et al.
(1992) Diagnostic problems with meningococcal disease in general practice. J Clin Epidemiol 45(11):1289–1293.
OpenUrl CrossRef PubMed
31.↵
1. Nadel S,
2. Britto J,
3. Booy R,
4. et al.
(1998) Avoidable deficiencies in the delivery of health care to children with meningococcal disease. J Accid Emerg Med 15(5):298–303.
OpenUrl Abstract/FREE Full Text
32.↵
1. Najaf-Zadeh A,
2. Dubos F,
3. Aurel M,
4. Martinot A
(2008) Epidemiology of malpractice lawsuits in paediatrics. Acta Paediatr 97(11):1486–1491.
OpenUrl CrossRef PubMed
33.↵
1. Najaf-Zadeh A,
2. Dubos F,
3. Pruvost I,
4. et al.
(2011) Epidemiology and aetiology of paediatric malpractice claims in France. Arch Dis Child 96(2):127–130.
OpenUrl Abstract/FREE Full Text
34.↵
1. Thompson M,
2. Mayon-White R,
3. Harnden A,
4. et al.
(2008) Using vital signs to assess children with acute infections: a survey of current practice. Br J Gen Pract doi:10.3399/bjgp08X279689.
OpenUrl Abstract/FREE Full Text
35.↵
1. Verbakel JY,
2. Van den Bruel A,
3. Thompson M,
4. et al.
(2013) How well do clinical prediction rules perform in identifying serious infections in acutely ill children across an international network of ambulatory care datasets? BMC Med 11:10.
OpenUrl CrossRef PubMed
36.↵
1. Moons KG,
2. Kengne AP,
3. Grobbee DE,
4. et al.
(2012) Risk prediction models: II. External validation, model updating, and impact assessment. Heart 98(9):691–698.
OpenUrl Abstract/FREE Full Text
37.↵
1. Cals JW,
2. Butler CC,
3. Hopstaken RM,
4. et al.
(2009) Effect of point of care testing for C reactive protein and training in communication skills on antibiotic use in lower respiratory tract infections: cluster randomised trial. BMJ 338:b1374.
OpenUrl Abstract/FREE Full Text
38.
1. Cals JW,
2. Chappin FH,
3. Hopstaken RM,
4. et al.
(2010) C-reactive protein point-of-care testing for lower respiratory tract infections: a qualitative evaluation of experiences by GPs. Fam Pract 27(2):212–218.
OpenUrl CrossRef PubMed
39.↵
1. Van den Bruel A,
2. Thompson MJ,
3. Haj-Hassan T,
4. et al.
(2011) Diagnostic value of laboratory tests in identifying serious infections in febrile children: systematic review. BMJ 342:d3082.
OpenUrl Abstract/FREE Full Text

In this issue

Download PDF

Download PowerPoint

Email Article

Citation Tools

Keywords

More in this TOC Section

Show more Research

Cited By...

Intended for Healthcare Professionals

[1] 1.↵
Moll van Charante EP,
van Steenwijk-Opdam PC,
Bindels PJ
(2007) Out-of-hours demand for GP care and emergency services: patients’ choices and referrals by general practitioners and ambulance services. BMC Fam Pract 8:46.
OpenUrl CrossRef PubMed

[2] Moll van Charante EP,

[3] van Steenwijk-Opdam PC,

[4] Bindels PJ

[5] 2.↵
Bruijnzeels MA,
Foets M,
van der Wouden JC,
et al.
(1998) Everyday symptoms in childhood: occurrence and general practitioner consultation rates. Br J Gen Pract 48(426):880–884.
OpenUrl Abstract/FREE Full Text

[6] Bruijnzeels MA,

[7] Foets M,

[8] van der Wouden JC,

[9] et al.

[10] 3.↵
Bateman SL,
Seed PC
(2010) Procession to pediatric bacteremia and sepsis: covert operations and failures in diplomacy. Pediatrics 126(1):137–150.
OpenUrl Abstract/FREE Full Text

[11] Bateman SL,

[12] Seed PC

[13] 4.
Prayle A,
Atkinson M,
Smyth A
(2011) Pneumonia in the developed world. Paediatr Respir Rev 12(1):60–69.
OpenUrl CrossRef PubMed

[14] Prayle A,

[15] Atkinson M,

[16] Smyth A

[17] 5.↵
Saez-Llorens X,
McCracken GH Jr.
(2003) Bacterial meningitis in children. Lancet 361(9375):2139–2148.
OpenUrl CrossRef PubMed

[18] Saez-Llorens X,

[19] McCracken GH Jr.

[20] 6.↵
Van den Bruel A,
Aertgeerts B,
Bruyninckx R,
et al.
(2007) Signs and symptoms for diagnosis of serious infections in children: a prospective study in primary care. Br J Gen Pract 57(540):538–546.
OpenUrl Abstract/FREE Full Text

[21] Van den Bruel A,

[22] Aertgeerts B,

[23] Bruyninckx R,

[24] et al.

[25] 7.↵
Thompson MJ,
Harnden A,
Del Mar C
(2009) Excluding serious illness in feverish children in primary care: restricted rule-out method for diagnosis. BMJ 338:b1187.
OpenUrl FREE Full Text

[26] Thompson MJ,

[27] Harnden A,

[28] Del Mar C

[29] 8.↵
Berger MY,
Boomsma LJ,
Albeda FW,
et al.
(2008) Guideline. Children with fever, [NHG-standaard Kinderen met koorts] http://nhg.artsennet.nl/kenniscentrum/k_richtlijnen/k_nhgstandaarden/Samenvattingskaartje-NHGStandaard/M29_svk.htm (accessed 13 Feb 2014).

[30] Berger MY,

[31] Boomsma LJ,

[32] Albeda FW,

[33] et al.

[34] 9.↵
National Institute for Health and Care Excellence
(May, 2013) Feverish illness in children, http://guidance.nice.org.uk/CG160 (accessed 27 Nov 2014).

[35] National Institute for Health and Care Excellence

[36] 10.↵
Reilly BM,
Evans AT
(2006) Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med 144(3):201–209.
OpenUrl CrossRef PubMed

[37] Reilly BM,

[38] Evans AT

[39] 11.↵
Maguire JL,
Kulik DM,
Laupacis A,
et al.
(2011) Clinical prediction rules for children: a systematic review. Pediatrics 128(3):e666–e677.
OpenUrl Abstract/FREE Full Text

[40] Maguire JL,

[41] Kulik DM,

[42] Laupacis A,

[43] et al.

[44] 12.↵
Van den Bruel A,
Haj-Hassan T,
Thompson M,
et al.
(2010) Diagnostic value of clinical features at presentation to identify serious infection in children in developed countries: a systematic review. Lancet 375(9717):834–845.
OpenUrl CrossRef PubMed

[45] Van den Bruel A,

[46] Haj-Hassan T,

[47] Thompson M,

[48] et al.

[49] 13.↵
Thompson M,
Van den Bruel A,
Verbakel J,
et al.
(2012) Systematic review and validation of prediction rules for identifying children with serious infections in emergency departments and urgent-access primary care. Health Technol Assess 16(15):1–100.
OpenUrl CrossRef PubMed

[50] Thompson M,

[51] Van den Bruel A,

[52] Verbakel J,

[53] et al.

[54] 14.↵
Oostenbrink R,
Thompson M,
Steyerberg EW,
ERNIE members
(2012) Barriers to translating diagnostic research in febrile children to clinical practice: a systematic review. Arch Dis Child 97(7):667–672.
OpenUrl Abstract/FREE Full Text

[55] Oostenbrink R,

[56] Thompson M,

[57] Steyerberg EW,

[58] ERNIE members

[59] 15.↵
Elshout G,
van Ierland Y,
Bohnen AM,
et al.
(2013) Alarm signs and antibiotic prescription in febrile children in primary care: an observational cohort study. Br J Gen Pract doi:10.3399/bjgp13X669158.
OpenUrl Abstract/FREE Full Text

[60] Elshout G,

[61] van Ierland Y,

[62] Bohnen AM,

[63] et al.

[64] 16.↵
National Institute for Health and Care Excellence
(May, 2007) Feverish illness in children — Assessment and initial management in children younger than 5 years, http://www.nice.org.uk/CG047 (accessed 27 Nov 2014).

[65] National Institute for Health and Care Excellence

[66] 17.↵
Muris JWM
(1990) Lung auscultation in general practice. A literature survey. [in Dutch]. Huisarts Wet 33(7):258–262.
OpenUrl

[67] Muris JWM

[68] 18.↵
Thompson M,
Coad N,
Harnden A,
et al.
(2009) How well do vital signs identify children with serious infections in paediatric emergency care? Arch Dis Child 94(11):888–893.
OpenUrl Abstract/FREE Full Text

[69] Thompson M,

[70] Coad N,

[71] Harnden A,

[72] et al.

[73] 19.↵
Pantell RH,
Newman TB,
Bernzweig J,
et al.
(2004) Management and outcomes of care of fever in early infancy. JAMA 291(10):1203–1212.
OpenUrl CrossRef PubMed

[74] Pantell RH,

[75] Newman TB,

[76] Bernzweig J,

[77] et al.

[78] 20.↵
Bleeker SE,
Derksen-Lubsen G,
Grobbee DE,
et al.
(2007) Validating and updating a prediction rule for serious bacterial infection in patients with fever without source. Acta Paediatr 96(1):100–104.
OpenUrl PubMed

[79] Bleeker SE,

[80] Derksen-Lubsen G,

[81] Grobbee DE,

[82] et al.

[83] 21.↵
Berger RM,
Berger MY,
van Steensel-Moll HA,
et al.
(1996) A predictive model to estimate the risk of serious bacterial infections in febrile infants. Eur J Pediatr 155(6):468–473.
OpenUrl CrossRef PubMed

[84] Berger RM,

[85] Berger MY,

[86] van Steensel-Moll HA,

[87] et al.

[88] 22.↵
Young Infants Clinical Signs Study Group (YICSSG)
(2008) Clinical signs that predict severe illness in children under age 2 months: a multicentre study. Lancet 371(9607):135–142.
OpenUrl CrossRef PubMed

[89] Young Infants Clinical Signs Study Group (YICSSG)

[90] 23.↵
Brent AJ,
Lakhanpaul M,
Thompson M,
et al.
(2011) Risk score to stratify children with suspected serious bacterial infection: observational cohort study. Arch Dis Child 96(4):361–367.
OpenUrl Abstract/FREE Full Text

[91] Brent AJ,

[92] Lakhanpaul M,

[93] Thompson M,

[94] et al.

[95] 24.↵
Buntinx F,
Mant D,
Van den Bruel A,
et al.
(2011) Dealing with low-incidence serious diseases in general practice. Br J Gen Pract doi:10.3399/bjgp11X548974.
OpenUrl Abstract/FREE Full Text

[96] Buntinx F,

[97] Mant D,

[98] Van den Bruel A,

[99] et al.

[100] 25.↵
van Ierland Y,
van Veen M,
Huibers L,
et al.
(2011) Validity of telephone and physical triage in emergency care: the Netherlands Triage System. Fam Pract 28(3):334–341.
OpenUrl

[101] van Ierland Y,

[102] van Veen M,

[103] Huibers L,

[104] et al.

[105] 26.↵
Thomson S,
Osborn R,
Squires D,
Reed SJ
, eds (2011) International profiles of health care systems (The Commonwealth Fund, New York, NY).

[106] Thomson S,

[107] Osborn R,

[108] Squires D,

[109] Reed SJ

[110] 27.↵
Donders AR,
van der Heijden GJ,
Stijnen T,
Moons KG
(2006) Review: a gentle introduction to imputation of missing values. J Clin Epidemiol 59(10):1087–1091.
OpenUrl CrossRef PubMed

[111] Donders AR,

[112] van der Heijden GJ,

[113] Stijnen T,

[114] Moons KG

[115] 28.↵
Thompson MJ,
Ninis N,
Perera R,
et al.
(2006) Clinical recognition of meningococcal disease in children and adolescents. Lancet 367(9508):397–403.
OpenUrl CrossRef PubMed

[116] Thompson MJ,

[117] Ninis N,

[118] Perera R,

[119] et al.

[120] 29.
Riordan FA,
Thomson AP,
Sills JA,
Hart CA
(1996) Who spots the spots? Diagnosis and treatment of early meningococcal disease in children. BMJ 313(7067):1255–1256.
OpenUrl FREE Full Text

[121] Riordan FA,

[122] Thomson AP,

[123] Sills JA,

[124] Hart CA

[125] 30.
Sorensen HT,
Moller-Petersen J,
Krarup HB,
et al.
(1992) Diagnostic problems with meningococcal disease in general practice. J Clin Epidemiol 45(11):1289–1293.
OpenUrl CrossRef PubMed

[126] Sorensen HT,

[127] Moller-Petersen J,

[128] Krarup HB,

[129] et al.

[130] 31.↵
Nadel S,
Britto J,
Booy R,
et al.
(1998) Avoidable deficiencies in the delivery of health care to children with meningococcal disease. J Accid Emerg Med 15(5):298–303.
OpenUrl Abstract/FREE Full Text

[131] Nadel S,

[132] Britto J,

[133] Booy R,

[134] et al.

[135] 32.↵
Najaf-Zadeh A,
Dubos F,
Aurel M,
Martinot A
(2008) Epidemiology of malpractice lawsuits in paediatrics. Acta Paediatr 97(11):1486–1491.
OpenUrl CrossRef PubMed

[136] Najaf-Zadeh A,

[137] Dubos F,

[138] Aurel M,

[139] Martinot A

[140] 33.↵
Najaf-Zadeh A,
Dubos F,
Pruvost I,
et al.
(2011) Epidemiology and aetiology of paediatric malpractice claims in France. Arch Dis Child 96(2):127–130.
OpenUrl Abstract/FREE Full Text

[141] Najaf-Zadeh A,

[142] Dubos F,

[143] Pruvost I,

[144] et al.

[145] 34.↵
Thompson M,
Mayon-White R,
Harnden A,
et al.
(2008) Using vital signs to assess children with acute infections: a survey of current practice. Br J Gen Pract doi:10.3399/bjgp08X279689.
OpenUrl Abstract/FREE Full Text

[146] Thompson M,

[147] Mayon-White R,

[148] Harnden A,

[149] et al.

[150] 35.↵
Verbakel JY,
Van den Bruel A,
Thompson M,
et al.
(2013) How well do clinical prediction rules perform in identifying serious infections in acutely ill children across an international network of ambulatory care datasets? BMC Med 11:10.
OpenUrl CrossRef PubMed

[151] Verbakel JY,

[152] Van den Bruel A,

[153] Thompson M,

[154] et al.

[155] 36.↵
Moons KG,
Kengne AP,
Grobbee DE,
et al.
(2012) Risk prediction models: II. External validation, model updating, and impact assessment. Heart 98(9):691–698.
OpenUrl Abstract/FREE Full Text

[156] Moons KG,

[157] Kengne AP,

[158] Grobbee DE,

[159] et al.

[160] 37.↵
Cals JW,
Butler CC,
Hopstaken RM,
et al.
(2009) Effect of point of care testing for C reactive protein and training in communication skills on antibiotic use in lower respiratory tract infections: cluster randomised trial. BMJ 338:b1374.
OpenUrl Abstract/FREE Full Text

[161] Cals JW,

[162] Butler CC,

[163] Hopstaken RM,

[164] et al.

[165] 38.
Cals JW,
Chappin FH,
Hopstaken RM,
et al.
(2010) C-reactive protein point-of-care testing for lower respiratory tract infections: a qualitative evaluation of experiences by GPs. Fam Pract 27(2):212–218.
OpenUrl CrossRef PubMed

[166] Cals JW,

[167] Chappin FH,

[168] Hopstaken RM,

[169] et al.

[170] 39.↵
Van den Bruel A,
Thompson MJ,
Haj-Hassan T,
et al.
(2011) Diagnostic value of laboratory tests in identifying serious infections in febrile children: systematic review. BMJ 342:d3082.
OpenUrl Abstract/FREE Full Text

[171] Van den Bruel A,

[172] Thompson MJ,

[173] Haj-Hassan T,

[174] et al.

Main menu

User menu

Search

Translation of clinical prediction rules for febrile children to primary care practice: an observational cohort study

Abstract

INTRODUCTION

METHOD

Study design

Study setting and selection of patients

How this fits in

Extraction of relevant clinical signs

Selection of CPRs and translation to the primary out-of-hours care population

Missing data

Statistical analyses

RESULTS

Description of the GPC-population

Performance of CPRs applied to the GPC-population

DISCUSSION

Summary

Strengths and limitations

Comparison with existing literature

Implications for research and practice

Acknowledgments

Appendix 1. Grouping of alarm signs for serious infection

Appendix 2. Electronic search strategy for clinical prediction rules

Appendix 3. Distribution of contacts over the percentiles of the predicted risk and frequency of referral within groups

Notes

Funding

Ethical approval

Provenance

Competing interests

Discuss this article

REFERENCES

In this issue

Citation Manager Formats

Jump to section

Keywords

More in this TOC Section

Related Articles

Cited By...

NAVIGATE

RCGP

MY ACCOUNT

NEWS AND UPDATES

AUTHORS & REVIEWERS

CUSTOMER SERVICES

CONTRIBUTE

CONTACT US