Abstract
Background Consultations for self-limiting infections in children are increasing. It has been proposed that digital technology could be used to enable parents’ decision making in terms of self-care and treatment seeking.
Aim To evaluate the evidence that digital interventions facilitate parents deciding whether to self-care or seek treatment for acute illnesses in children.
Design and setting Systematic review of studies undertaken worldwide.
Method Searches of MEDLINE and EMBASE were made to identify studies (of any design) published between database inception and January 2019 that assessed digital interventions for parents of children (from any healthcare setting) with acute illnesses. The primary outcome of interest was whether the use of digital interventions reduced the use of urgent care services.
Results Three studies were included in the review. They assessed two apps and one website: Children’s On-Call — a US advice-only app; Should I See a Doctor? — a Dutch self-triage app for any acute illness; and Strategy for Off-Site Rapid Triage (SORT) for Kids — a US self-triage website for influenza-like illness. None of the studies involved parents during intervention development and it was shown that many parents did not find the two apps easy to use. The sensitivity of self-triage interventions was 84% for Should I See a Doctor? compared with nurse triage, and 93.3% for SORT for Kids compared with the need for emergency-department intervention; however, both had lower specificity (74% and 13%, respectively). None of the interventions demonstrated reduced use of urgent-care services.
Conclusion There is little evidence to support the use of digital interventions to help parent and/or carers looking after children with acute illness. Future research should involve parents during intervention development, and adequately powered trials are needed to assess the impact of such interventions on health services and the identification of children who are seriously ill.
INTRODUCTION
Acute illnesses in children are a common reason why parents seek urgent care, and the rate of acute admissions with self-limiting infections for young children has been increasing year on year since 1999.1,2 A small number of acute illnesses can become severe, but the vast majority are self-limiting and can be effectively managed in the community if parents feel able to provide adequate care at home and can access professional advice when needed.
In the UK, NHS 111 — the telephone triage helpline, introduced in 2013 — has been criticised for being very risk averse: the vast majority of recommendations for children aged <5 years are to seek primary care (80%) or urgent review in an emergency department (ED) (10%); in only 10% of cases is self-care recommended.3 Although it was hoped that NHS 111 would be part of the solution to the increase in paediatric emergency admission rates, which has been labelled a systematic failure of the NHS,2 data suggest that it has had a mixed impact.4,5 Children with self-limiting acute illnesses presenting to urgent care are highly likely to be prescribed antibiotics (often unnecessarily),6 which could contribute to the global crisis in antibiotic resistance, cause unnecessary side-effects for the child, and encourage parents to feel it is necessary to seek urgent care next time their child has a self-limiting illness. Access to urgent care for children with illnesses that do warrant urgent attention may also be delayed.
Alongside a rise in the availability of smartphone technology and internet access globally, national surveys have found that more than half of UK adults will research health topics via their mobile phones prior to seeking medical care.7 The NHS Long Term Plan highlights digital and mobile technology as an opportunity to support parents/guardians and carers in making the best decisions when considering accessing health care for their children by:
providing evidence-based advice to prevent inappropriate treatment seeking for children with minor illnesses; and
signposting those with signs of severe illness to urgent-care services.8,9
As such, it is becoming increasingly recognised that optimising the utilisation of health technology is essential to creating an effective, modern healthcare system.10 This review aimed to summarise the current evidence for using digital technology to enable parents to make better decisions regarding self-care and treatment seeking for acute illnesses in children.
It is widely believed that digital interventions will play an important role in the future delivery of health care. Parents of children with non-serious, self-limiting illnesses have demonstrated that they need reassurance and self-care information to help them decide if, and when, to seek health care. The findings presented here highlight an absence of evidence regarding the effectiveness of digital interventions, and low levels of satisfaction if they are developed without the involvement of their intended users. |
METHOD
This review follows the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines,11 as well as those from the Cochrane Collaboration.12 The article was registered on the PROSPERO database (registration no: CRD42019127125).
Data sources, search strategy, and selection criteria
MEDLINE and EMBASE were used in the literature search to identify articles published from inception until 19 January 2019. Search terms (see Supplementary Figure S1) were developed in collaboration with a librarian and adapted for each database. MeSH terms were exploded (searches retrieved all references indexed to a term, as well as any narrower term[s]) in order to capture as many acute diseases as possible. Two authors independently screened the titles and abstracts of all articles identified. Full-text copies of articles that appeared to be eligible were assessed for inclusion and their reference lists were screened.
Articles were eligible if they reported on studies (all designs) that investigated the use of digital interventions by parents of children with acute illnesses. Articles could be published in any language; children could be of any age and recruited from any healthcare setting. Studies that included adults were eligible for inclusion if children had also been included. Acute illnesses may have been infectious or non-infectious in origin.
The primary outcome of interest was whether the use of digital interventions reduced the use of urgent-care services (number of consultations per patient). Secondary outcomes included:
ease of use;
user satisfaction;
sensitivity/specificity of triage advice;
incidence of adverse events;
use of antibiotics; and
cost effectiveness.
Data extraction and quality assessment
Data extraction and risk of bias assessment were undertaken independently and in duplicate by the two authors who screened the titles and abstracts. The data-extraction form was specifically designed and piloted for this review. Risk of bias was assessed using a standardised form according to study type, devised by the National Heart, Blood and Lung Institute13 (see Supplementary Table S1).
RESULTS
In total, 1767 articles of potential interest were identified after the removal of duplicates. Full-text review was undertaken for 21 articles and references were cross-checked, which revealed one further relevant article (Figure 1). Excluded articles are listed (see Supplementary Table S2). Three studies met the inclusion criteria: one pilot randomised controlled trial (RCT),14 one prospective cross-sectional study,15 and one pilot cohort study.16
Study characteristics
The three studies reported on 4848 participants; full study characteristics are outlined in Table 1. On risk of bias assessment, Anhang Price et al’s cohort study16 was rated fair, while the other two studies (Lepley et al 14 and Verzantvoort et al) 15 were rated poor (see Supplementary Table S1). Lepley et al 14 studied the acceptability of Children’s On-Call, an advice-only mobile app that uses information from the Barton Schmidt Paediatric Telephone Advice manual17 and is derived from paediatric clinical protocols. It contains answers to health questions that are commonly asked by parents. Acceptability was compared with written/video information in a feasibility RCT of 98 parents (of children aged <11 years) presenting to the paediatric ED with any non-urgent illness in 2014.
Anhang Price et al 16 focused on Strategy for Off-site Rapid Triage (SORT) for Kids, a self-triage website developed by the Centre for Disease Control and Prevention and the American Academy of Pediatrics. Although originally intended to be made freely available to the public during the 2009 influenza (flu) season, it was decided that a pilot study was needed first to assess the website’s safety. Anhang Price et al 16 studied its acceptability and accuracy in two paediatric EDs in 2012; the study involved 294 children (aged 0–18 years) presenting with influenza-like illness, recruited by ED triage nurses.
Verzantvoort et al 15 studied the acceptability and accuracy of the self-triage mobile app, Should I See a Doctor?. This was developed by a Dutch GP out-of-hours (OOH) clinic, based on the Dutch College of GPs’ triage system and validated by the Scientific Institute for Quality of Healthcare. The study involved 4456 patients in 2014–2015 (of whom 11.9% were aged 0–12 years).
Study outcomes
Use of urgent-care services
Lepley et al 14 compared non-urgent ED attendance rates between groups over 6 months following the introduction of Children’s On-Call, using both formal chart review and parental self-report. No statistically significant differences were observed between groups.14
Sixty-five percent of those who used the Should I See a Doctor? app intended to follow its advice; significantly higher for patients aged <13 years (odds ratio [OR] 1.8, 95% confidence interval [CI] = 1.3 to 2.3), of male sex (OR 1.2, 95% CI = 1.1 to 11.4), and who were satisfied with the app (OR 2.5, 95% CI = 2.2 to 2.9) (data not shown). This intention was lower among those receiving wait-and-see advice (56%), compared with those who were advised to contact their GP during the day (75%), contact OOH services (61%), or were given self-care advice (67%)14 (Table 2).
Uptake, acceptability, and satisfaction with the intervention
There was low usage for the Children’s On-Call app and parents statistically significantly preferred the book (containing written health advice) they were offered, which came with an introductory video (Table 2). Of parents allocated to the app, 57.1% and 35.1% downloaded and used it, respectively (Table 3), whereas 73.0% of parents allocated to the book/video group used the book (data not shown). Parents and/or carers were also statistically significantly more likely to recommend the book to friends (100.0% versus 48.7%), found it easier to understand (94.6% versus 26.0%), and considered it more useful (70.3% versus 37.8%) than the app (data not shown). These findings were reflected in the qualitative analysis of open comments among parents with both low and adequate health literacy. However, in per-protocol analysis, comparing groups of those who were followed up on ≥1 occasion(s), there was no statistically significant difference in use (P = 0.530), understanding (P = 0.222), recommendations (P = 0.517), or usefulness (P = 0.983) of the app compared with the book.
Should I See a Doctor? was rated ‘very clear’ or ‘clear’ by 63.9% of users and 55.7% were ‘very satisfied’ or ‘satisfied’ (Table 3); however, satisfaction was statistically significantly higher among parents of patients aged 0–12 years compared to patients aged >12 years (OR 0.7, 95% CI = 0.55 to 0.89) (data not shown).
Ninety percent of participants found the SORT for Kids website ‘very easy’ to use, which was independent of race, ethnicity, or educational attainment. No data were collected on patients’ satisfaction with the result of their website triage, as they were blinded to that result. To ensure that using the website did not influence the parent or healthcare provider’s subsequent decision-making, the website gave no feedback regarding the child’s risk status and made no recommendations. The aim of the study was to evaluate the accuracy and safety of the triage but not to actually use the triage result to influence the patients’ or the clinicians’ treatment choices.
Accuracy of triage
The SORT for Kids (website) and Should I See a Doctor? (app) offered self-triage functionality. The sensitivity and specificity of Should I See a Doctor? was compared with nurse triage and self-triage combined in 126 (2.8%) of the 4456 users who received additional telephone-based nurse triage. In 81% of cases, the app’s advice corresponded to the outcome of the nurse triage, with sensitivity of 84%, specificity of 74% (Table 4), and positive- and negative-predictive values of 88% and 67%, respectively (data not shown).15 For 8% and 11% of cases, respectively, the app over- and underestimated symptom risk (data not shown); however, in no cases of undertriage were the symptoms considered to be life threatening.
SORT for Kids correctly identified 14 of the 15 children in whom an ED visit was deemed necessary by ED clinicians (93.3% sensitivity) (Table 4). It also identified all eight children who returned to the ED with similar symptoms within 7 days as high-risk (100% sensitivity) (data not shown). However, the algorithm had a very low specificity; it correctly classified as ‘low/intermediate risk’ only 35 of the 271 children whose visit was deemed unnecessary (12.9%, 95% CI = 9.2 to 17.5) (data not shown).
Other outcomes
None of the articles reported antibiotic use, incidence of adverse events (hospitalisation, mortality), or cost effectiveness.
DISCUSSION
Summary
There is a lack of evidence to support using digital interventions to advise parents on when to self-care or seek treatment to manage acute illness in children. Neither the Chldren’s On-Call or Should I See a Doctor? apps reported a reduction in the use of urgent-care services (a secondary outcome for the study on Children’s On-Call).
Usability of the SORT for Kids website (self-triage) was good, but usability and satisfaction with Should I See a Doctor? (self-triage app) and Children’s On-Call (advice-only app) was modest to poor. The majority of those who used Should I See a Doctor? stated that they intended to follow the app’s advice, with that intention being lowest among those receiving advice to ‘wait and see’. However, the proportion of users who actually heeded advice was not reported.
With regards to the accuracy of triage, the sensitivity of both self-triage interventions was good, but at the cost of specificity for SORT for Kids.
None of the digital interventions were developed with input from intended users, which may explain their poor acceptability. Furthermore, as users of SORT for Kids and Children’s On-Call were recruited in the ED, the results may not necessarily be generalisable to the general population. In addition, there was no evidence that parents would use these digital interventions for future episodes of illness in the home environment. For Children’s On-Call, it is inferred that patients would not use it as they rated the app less ‘useful’ than the booklet. For SORT for Kids, participants were not asked if they would continue to use the website.
Strengths and limitations
A comprehensive search was conducted using many different terms for acute childhood illnesses, with no restrictions on language or study type. However, the grey literature was not searched due to a lack of resources, and the lack of standardised keywords and MeSH terms means that a few potentially eligible articles may have been missed. The acute illnesses listed in the search terms were not exhaustive, so it is possible that some studies may have been missed if they reported on specific illnesses not picked up in the search. However, an attempt was made to mitigate this by enlisting the help of a librarian, using additional broad search terms, and exploding MeSH terms. The small number of included studies and heterogeneity of study populations, interventions, and outcome measures meant that drawing comparisons was difficult.
Comparison with existing literature
This is the first systematic review of digital interventions as a tool to enable parents to make better decisions on self-care and treatment seeking for children with acute illnesses. A previous review investigated the effects of paper-based interventions, demonstrating that parental help-seeking behaviour can be modified to enable parents to self-care, resulting in lower rates of consultation.18
Online interventions for acute illness have been shown to modify health-seeking behaviour without increasing hospital admissions in adults.19 A recent audit of self-triage apps (only one of which — Healthy Children [US] — was specifically for children) identified that triage advice from symptom checkers is generally risk averse, prioritising sensitivity over specificity and encouraging users to seek professional help for conditions when self-care is reasonable.20 Similarly, a recent evidence synthesis of different models of urgent-care delivery 21 concluded that telephone triage was safe at the level of the individual, but came at the cost of efficiency; some studies suggested that nurses were more likely to refer to higher-level care than doctors.21
Digital interventions aimed at parents and carers of children have been successful at promoting self-care for chronic conditions such as eczema, diabetes, and asthma,22–23 especially when developed using a person-based approach with input from intended users,24 and it has been recognised that good-quality, accessible information is key in empowering children and carers to self-manage their long-term conditions.25 However, the clinical utility of such apps (compared with simply providing written information in a booklet) remains uncertain;23,26 a recent review found insufficient evidence to support the efficacy of apps directed at older children with chronic mental-health problems.27 Written advice in the form of a booklet has been shown to help parents decide when they can confidently self-care and when they need to seek advice from a health professional.28–31
It has been well documented that parents with lower health literacy are more likely to seek OOH health care unnecessarily for non-urgent complaints;32,33 a recent systematic review highlighted that mobile health apps may be of particular benefit in this group and have the potential to reduce disparities in health care.34 Lepley et al have noted that the use of jargon and complex sentences in Children’s On-Call may have been responsible for its poor acceptability, particularly given that more than half of the study population had low health literacy.14 Again, these issues may be improved by involving intended end users in the development process to ensure that the intervention is not only acceptable, usable, and effective, but also engaging and persuasive.24 It is also important to consider how, and by whom, future digital interventions are delivered; if they are not delivered effectively, are not easy to access, or not supported by empathetic, engaging staff when parents are made aware of them they may end up being ineffective.
Implications for research
The findings presented here highlight the need for rigorous evaluation of digital interventions and the need to develop those interventions in collaboration with their intended target populations, for example, through the person-based approach advocated by Yardley et al.24 There are many examples of such an approach resulting in the development of cost-effective digital interventions — one such is Internet Doctor, an interactive UK-based website for the self-management of respiratory infections, which was shown to reduce contact with doctors without increasing hospital admissions.19 Self-care advice in mobile health apps (for which this review showed acceptability and the intention to follow advice to be modest) also needs to be optimised for the target population, with further links given to endorsed websites and educational resources.15,35
Another priority for research is to develop triage algorithms that have sensitivity to detect serious illness, yet also have a good specificity; this would correctly identify those in need of urgent care and avoid sending patients to urgent-care services unnecessarily. At present, most algorithms are very risk averse and rarely promote self-care or watchful waiting, resulting in unnecessary consultations, antibiotic overuse, and delays for those for whom urgent care is warranted. Once appropriate (that is, user-friendly, sensitive, and specific) self-triage interventions have been developed, adequately powered RCTs should assess their impact on healthcare resource use and patient outcomes.
There is huge scope to empower parents to provide home care for children who have self-limiting acute illnesses. Based on current evidence, the authors are unable to recommend any digital interventions as a support tool to enable parents to make better decisions about self-care and treatment seeking for acute illnesses in children and thus reduce urgent-care service use. Future interventions should be developed in collaboration with their target audience to improve usability and satisfaction; in addition, algorithms resulting in greater specificity should be developed to avoid unnecessary use of urgent-care services, while maintaining sensitivity to correctly identify children with serious illnesses.
Acknowledgments
The authors would like to thank Paula Sands from the University Hospital Southampton library for her assistance with developing the search terms used in this review.
Notes
Funding
The salary of Merlin L Willcox was funded by the National Institute for Health Research (NIHR), under grant: CL-2016-26-005. Emily Donovan is an NIHR-funded academic clinical fellow.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors have declared no competing interests.
Discuss this article
Contribute and read comments about this article: bjgp.org.uk/letters
- Received June 19, 2019.
- Revision requested July 29, 2019.
- Accepted August 7, 2019.
- © British Journal of General Practice 2020