Assessing acutely ill children in general practice using the National PEWS and LqSOFA clinical scores: a retrospective cohort study

Abstract


Introduction
(3)(4)(5)(6) The assessment of acutely unwell children can be challenging in general practice due to vague systemic symptoms and the low prevalence of serious illness. (6)It can be difficult for general practitioners (GPs) to identify children who can be safely managed at home while also identifying the few who are at risk of serious illness and need hospital admission.
Various clinical scoring systems have been developed to help clinicians.The National Institute for Health and Care Excellence (NICE) 'Traffic Light' system was recommended for use in general practice but, until recently, had not been validated in this setting.It has now been found to perform poorly on account of categorising almost all children as 'moderate' or 'high' risk of serious illness and displaying low sensitivity and specificity. (7)Clinical prediction tools developed in hospital settings may also perform well in general practice, but it is essential that these are tested and validated in general practice prior to implementation.
One promising tool developed in a paediatric Emergency Department (ED), is the Liverpool quick Sequential Organ Failure Assessment (LqSOFA) score, developed to identify lifethreatening infections in febrile children. (8)This score has a good prognostic ability for detecting critical care admissions, consisting of four variables: heart rate, respiratory rate, consciousness level, and capillary refill time.
Most hospitals throughout the UK have a Paediatric Early Warning Score (PEWS) to identify children at risk of deterioration.A standardised 'National PEWS' has been developed by the Royal College of Paediatrics and Child Health (RCPCH), the Royal College of Nursing (RCN), and NHS England. (9)This score has recently been introduced in hospitals throughout England and it is hoped that this score or a modified version might be suitable for implementation in general practice. (9)If the new National PEWS could be adopted in general practice, it could bridge the gap in continuity between primary and secondary care, allowing a synergistic approach to the assessment of unwell children in a variety of settings.
Any clinical decision tools incorporated into general practice must be validated in this setting, to ensure clinicians understand the accuracy and utility of these scoring systems before using them to guide decisions.Such tools should have a high sensitivity, ensuring that all children with a serious illness are correctly 'flagged' and referred for secondary care assessment whilst providing reassurance that those who are not flagged can be safely managed at home.The aim of this study is to validate the LqSOFA score and National PEWS within general practice.

Methods
This study is a retrospective cohort study linking general practice study data with hospital admission data in England and Wales.

Study Participants
A secondary analysis of data from a previous study, the Diagnosis of Urinary Tract Infection in Young Children (DUTY) study, was performed. (10)The DUTY study was a prospective cohort study analysing the presenting signs and symptoms of acutely ill children aged under five years in primary care, to explore the features of urinary tract infections.The details of the DUTY study and the cohort demographics are reported elsewhere. (7,11) nly participants presenting to general practice were included in this study.12)

Scores undergoing validation
The LqSOFA score consists of four variables, each scoring one point if abnormal: heart rate, respiratory rate, consciousness level, and capillary refill time (Supplementary Table S1). (8)The National PEWS consists of four age-specific charts, with a maximum score total of 18: the charts for children aged 0-11 months (chart 1) and 1 to 5 years (chart 2) were used in this study (Supplementary Tables S2 and S3).

Score calculations
Each criterion was matched to the variables available within our general practice dataset.Children with missing data for two or more of the scoring variables were excluded.If only one component was missing the child was included and the variable was assumed to be normal.This approach was used in the original LqSOFA and previous National PEWS papers. (8,13) OFA score matching The matching of variables between the LqSOFA score and our dataset was discussed within the study team comprising of clinicians and senior researchers (Table S1).The variables 'heart rate' and 'respiratory rate' were directly mapped onto our data, and the matching of consciousness level was unanimously agreed upon.Our 'capillary refill time' (CRT) variable of 2-5 seconds could not be easily matched to the LqSOFA categories of < 3 seconds (normal) or ≥3 seconds (abnormal).We consulted seven general practitioners and seven secondary care clinicians; all of the general practitioners supported classifying '2 to 5 seconds' as abnormal, with four out of seven of the secondary care respondents in agreement.Therefore, we classified CRT 2-5 seconds as abnormal.
National PEWS score matching The matching of variables was agreed by the study team as described in Supplementary Table S4.We continued to classify CRT '2 to 5 seconds' as abnormal.We excluded the PEWS variables 'blood pressure' and 'oxygen requirement' as these were not available in our general practice data.

Outcome measures
Primary outcome Our primary outcome was a hospital admission within two days of the general practice index consultation during which the child was recruited for the DUTY study.A 'hospital admission' was defined as a spell in hospital as an inpatient under the care of a consultant; assessment in the ED was not coded as an admission unless the treating team decided to admit them.

Secondary outcome
Our secondary outcome was a composite outcome 'serious illness episode': either a serious illness diagnosed in hospital within two days of GP consultation or a hospital admission lasting at least one night, within two days of GP consultation.Our definition of 'serious illness' has been described previously, and was based on the NICE definition within their fever guidelines. (7,14)

Statistical analysis
The cohort was analysed descriptively to define the sample characteristics.This included general demographics (age, sex, number of days unwell, presence of fever, score totals), hospital admissions, and comparison of children admitted and not admitted to hospital.
The test performance of the LqSOFA and National PEWS was then assessed.For each child, a score total was calculated using each assessment system respectively, by adding up points for the constituent variables.PEWS totals were calculated separately using the age-specific charts of <12 months and 1-5 years.The age groups were then combined for analysis.
Area under the receiver operating characteristic curves (AUC) were calculated for each of the scores.Sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood values were calculated for each system, at each score threshold, alongside a 95% confidence interval (CI).Analyses were performed using SPSS V.25 and Stata v16.0.A sensitivity analysis was performed, restricting the cohort to febrile children, identified using the definition "measured or perceived elevation of body temperature above the normal daily variation (≥38ᵒC) by a parent or clinician".This was chosen to reflect the cohort characteristics of the original

Results
There were 7,163 children included in the original DUTY study.We excluded those not recruited from general practices (n=366); those where we were unable to link with hospital data (n=88); and those without any clinical variables recorded (n=6).This left a total of 6,703 children included in this study, whose demographics have been described previously. (7)FA Data for two or more variables were missing in 1,135 (16.9%) children and they were excluded from the analysis (Table S5).The most common missing variables were heart rate (22.7%) and respiratory rate (20.8%).Children excluded from the analysis were younger and less likely to be febrile although no significant difference in hospital admissions was seen (Table S6).5,568 (83.1%) children had either complete variables (n=4508) or one variable missing (n=1060), enabling an LqSOFA score to be calculated.The median age of included children was 2.2 years (Table 1).The majority (84%) of children scored 0 on the LqSOFA (Figure 1).The most common reason for scoring was a prolonged CRT, in 783 (14.1%) children, followed by reduced conscious level in 55 (1.0%) children.
Secondary outcome There were 42 (0.8%) children who had a serious illness episode.For this outcome, a score of at least 1 on the LqSOFA had a sensitivity of 23% (95% CI 14% to 39%), slightly worse than for the primary outcome, and a comparable specificity of 85% (84% to 85%, Table 2).Sensitivity and specificity values were similar when the population was limited to febrile children (Table 2).

National PEWS
Overall, 4,899 (73.1%) children were included in the analysis (Figure 2).There was a higher proportion of children <12 months missing data for two or more variables (32.2%,Table S5) compared to children >12 months (25.2%).The children excluded from the analysis were younger and less likely to be febrile, with no significant difference in hospital admissions (Table S6).The median age of included children was 7.2 months for children aged 12 months or younger (PEWS Chart 1) and 2.9 years for those older than 12 months (PEWS Chart 2; Table 1).
The most common PEWS score total was 1, accounting for 38.2% of score totals (Figure 1).A raised heart rate and raised respiratory rate were the most common reasons for scoring.
When the population was limited to febrile children, 74 (1.5%) were admitted to hospital.The National PEWS performed slightly less well, with a sensitivity of 79.7% (95% CI 69.2% to 87.3%) and specificity of 31.4% (29.9% to 32.9%) for a threshold score of >1 (Table 3).
Secondary outcome There were 38 (0.8%) children who had a serious illness episode.For this outcome, a National PEWS score of at least 1 had a sensitivity of 73.7% (95% CI 58.0% to 85.0%), slightly worse than for the primary outcome, and a specificity of 32.3% (31.0% to 33.7%, Table 3).Sensitivity and specificity values were similar when the population was limited to febrile children (Table 3).

Summary
Overall, the results demonstrate that neither the LqSOFA nor the National PEWS are accurate for identifying acutely unwell children admitted to hospital within two days of a general practice consultation.Both scores demonstrated poor discrimination for predicting hospital admissions, with an AUC range of 0.57 to 0.66.Neither tool performed well for identifying serious illnesses, with the confidence intervals overlapping such that the scoring systems were no better than chance.The LqSOFA score had a high specificity for a score of at least two (99.5%).This strong 'rule in' ability would highlight to GPs that children scoring at least two require urgent referral to hospital and should not be sent home.Conversely, the sensitivity is low; most children who were admitted would be missed using this threshold.An LqSOFA score of least one had a slightly improved sensitivity of 30.6% but a poorer specificity of 84.7%.For a scoring system to be useful to GPs it needs to have a high sensitivity, identifying all children who need admission so that those not flagged up can be confidently managed at home.The National PEWS had a better sensitivity than Lq-SOFA with a sensitivity of 81.0%, using a threshold of at least one point to 'flag' which children may require hospital admission.Although capturing the majority of seriously ill children, this would still miss 19% of children requiring admission.Specificity was low at 32.5%, and 68% of all presenting ill children would be flagged as needing admission.

Strengths and limitations of the study
This study is, to the authors' knowledge, the first to evaluate the performance of an adjusted National PEWS score using general practice data, providing important results regarding the accuracy of this tool if it were to be introduced into general practice.We utilised a large dataset of acutely unwell children presenting to general practice including detailed presenting symptoms and signs.
We were not able to match all variables from either scoring system.There were differences in the categorisation of CRT which may have resulted in a greater number of children scoring for this variable.We did not have data available for the PEWS' variables blood pressure and oxygen requirement; however, these measurements are not routinely performed in general practice and the inclusion of these would not represent usual clinical practice and would be of little use for GPs if included in this scoring system.We had to exclude 26.9% of children due to missing variables.This could have created a selection bias as excluded children were younger; however, there was no difference in hospital admission rates.
Hospital admissions amongst children can be influenced by a variety of contextual factors and do not always indicate illness severity.We did not have data available to allow exploration of biomedical or social reasons for admissions.
Additionally, our primary outcome of 'admission within two days' may have reduced the accuracy of these scoring systems; a shorter time frame of 24 hours is often used in hospitalbased predictive studies to identify early deterioration.Furthermore, the National PEWS displays trends in scores which can be used to track deterioration.Including these factors in our study could have improved the score's sensitivity.However, our primary outcome reflected the intervals used in the pre-existing LqSOFA and National PEWS studies.
Finally, our dataset may not have included extremely unwell children requiring immediate transfer to hospital from the community.Nevertheless, decision tools would not be needed for these occasions as it would be clear to the clinician that urgent admission is required.

Comparison with existing literature
We could not find other studies evaluating either of these scoring systems in a general practice setting.The LqSOFA derivation and validation cohorts included children attending a UK paediatric ED with a fever. (8)Their primary outcome was an admission to critical care within 48 hours.Using an LqSOFA score ≥1, they reported a sensitivity of 71.9% and specificity of 85.0 (AUC 0.81), demonstrating that the LqSOFA performs better in EDs than in general practice.This is likely due to the difference in the stages of presentation and prevalence of serious illness between the two settings.
The current National PEWS has only recently been rolled out in hospitals nationwide.There is one previous study that assessed an earlier version alongside six other regional PEWS, within an ED. (13)For the primary outcome of 'critical care admission within 48 hours' the National PEWS performed well, with sensitivities and specificities of 89.6% and 84.7% respectively, using a threshold of ≥5.No data were presented for lower score thresholds.We identified one scoring system developed and validated in primary care in Belgium.This 'fourstep decision tree' had a sensitivity of 100% and specificity of 83.6% when it was validated.
However, the validation study included children up to age 16 and from ED and outpatient settings.This warrants further exploration in a UK cohort of younger children presenting to general practice. (15,16) lications for research and practice The current NICE-recommended clinical tool used in general practice is the 'Traffic Light' system, which has poor sensitivity (58.8%) and specificity (68.5%) for identifying seriously ill children. (7)The two scores evaluated here are simpler to use; with objective variables in comparison to the many subjective variables in the Traffic Light system.However, neither performed well.
If the National PEWS had performed well in general practice, this could have provided a common language across pre-hospital and hospital settings, improved continuity of care, and potentially improved outcomes in children.However, our study has shown that an adapted version of the National PEWS performs poorly for predicting admissions from general practice within two days and should not be incorporated nationally into general practices for the assessment of acutely unwell children as it stands.It is possible that the full PEWS may have performed better than our adapted version, however, blood pressure is not commonly measured in children in general practice nor is oxygen commonly administered.
Further research is needed to derive and validate an accurate scoring system in general practice that is both easy to use and accurate.This may involve validation of an existing score or the development of a new or adjusted early warning score using prospective general practice data and including qualitative work with primary and secondary care clinicians.S7.
LqSOFA = Liverpool quick Sequential Organ Failure Assessment.AUC = area under receiver operating characteristic curve a For the secondary outcome, all percentages for predictive values are rounded to mask derivation of raw numbers which need to be suppressed due to small numbers; b serious illness episode is defined as either a serious illness diagnosed in hospital within two days of GP consultation or a hospital admission lasting at least one night, within two days of GP consultation .c Admissions Data could not be displayed due to unmasking of small numbers.S8 PEWS = Paediatric Early Warning Score.AUC = area under receiver operating characteristic curve.a For the secondary outcome, all percentages for predictive values are rounded to mask derivation of raw numbers which need to be suppressed due to small numbers; b serious illness episode is defined as either a serious illness diagnosed in hospital within two days of GP consultation or a hospital admission lasting at least one night, within two days of GP consultation. .c Admissions Data could not be displayed due to unmasking of small numbers.

FiguresFigure 1 :
Figures Figure 1: Graph demonstrating score frequencies for the LqSOFA and National PEWS.The majority of children are scoring either zero or one point.

Figure 2 :
Figure 2: Sample cohorts for National PEWS analysis

Table 3 : Combined National PEWS scoring system for all children <5 years and febrile children <5 years for primary and secondary outcomes.
Rounded figures used.Full table can be reviewed in Supplementary Table