Article Text
PDF
Abstract
Objectives Evaluation of a pilot study of an online consultation system in primary care. We describe who used the system, when and why, and the National Health Service costs associated with its use.
Design 15-month observational study.
Setting Primary care practices in South West England.
Results 36 General practices covering 396 828 patients took part in the pilot. The online consultation website was viewed 35 981 times over the pilot period (mean 9.11 visits per 1000 patients per month). 7472 patients went on to complete an ‘e-consultation’ (mean 2.00 online consultations per 1000 patients per month). E-consultations were mainly performed on weekdays and during normal working hours. Patient records (n=485) were abstracted for eight practices and showed that women were more likely to use e-consultations than men (64.7% vs 35.3%) and users had a median age of 39 years (IQR 30–50). The most common reason for an e-consultation was an administrative request (eg, test results, letters and repeat prescriptions (22.5%)) followed by infections/immunological issues (14.4%). The majority of patients (65.2%) received a response within 2 days. The most common outcome was a face-to-face (38%) or telephone consultation (32%). The former were more often needed for patients consulting about new conditions (OR 1.56, 95% CI 1.05 to 2.27, p=0.049). The average cost of a practice’s response to an e-consultation was £36.28, primarily triage time and resulting face-to-face/telephone consultations needed.
Conclusions Use of e-consultations is very low, particularly at weekends. Unless this can be improved, any impact on staff workload and patient waiting times is likely to be negligible. It is possible that use of e-consultations increases primary care workload and costs. Online consultation systems could be developed to improve efficiency both for staff and patients. These findings have implications for software developers as well as primary care services and policy-makers who are considering investing in online consultation systems.
- general practice
- primary care
- online consultations
- e-consultations
- evaluation
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Largest UK study to date examining use of a primary care online consultation system.
Benefits from a number of quantitative data sources (website usage statistics, subset of electronic patient records and public data on general practices).
We were unable to collect data on a control group and were therefore limited to a descriptive evaluation.
Background
There is an increasing demand for UK primary care services, with workload increasing by 16% between 2007 and 2014.1 General practices have struggled to meet this challenge, particularly in the context of a declining workforce,2 3 and difficulties accessing services have become a key source of patient dissatisfaction4 and practitioner stress.5
In response to this, the National Health Service (NHS) England report ‘General Practice Forward View’ promotes a greater use of technology by general practices, for example, via online consultation systems, as a potential way to meet demand.6 The Department of Health have said that by 2020, they will establish 7-day access to primary care, with the view that broadening access options may help meet demand.7 The Prime Minister’s Challenge Fund (now called the GP Access Fund) was set up in 2013 to ‘improve access to general practice and stimulate innovative ways of providing primary care services’.8 In 2017, NHS England is increasing Clinical Commissioning Groups budgets, specifically for the provision of general practice IT/technology services, and making available an additional £45 million over 3 years to support the uptake of online consultations.9
Communications and online technology (such as email, video and online consultations) are currently being investigated to improve patient access and make more efficient use of practitioners’ time. So far, the evidence about use and effectiveness of such technology is conflicting10–19 and few studies have been performed in the UK. Rigorous evaluation of online consultation systems is crucial before nationwide encouragement or roll-out of such systems. Practices will incur costs when introducing new systems (software fees, licences, staff time to learn and integrate the new system into their practice) and they need as much information as possible on acceptability, effectiveness and cost-effectiveness to make informed decisions about whether to invest in them. In the UK at least, the current policy could also have implications for the taxpayer. Also, in the context of other expensive, failed IT programmes in UK healthcare such as the partially developed and abandoned system of national electronic medical records,20 21 it is vital that other technological systems are evaluated as early as possible before they are implemented on a broad scale.
An online consultation system was piloted by 36 practices and a number of patient records were abstracted from 8 of these practices. We performed a mixed-methods observational evaluation, including quantitative, qualitative and health economic analyses. The objectives of this paper are to: (1) report on how often and when the online system was used (with e-consultations with a general practitioner being an option within that system), (2) summarise the types of users, the reasons for e-consultation and what happened as a result and (3) estimate the cost associated with e-consultations. It should be noted that here we report on the quantitative and health economic findings. Qualitative findings will be reported separately.
Methods
The pilot
In 2014, the GP Access Fund supported a consortium of general practices in South West England22 to pilot an online consultation system called ‘eConsult’ (previously called WebGP).23 eConsult (hereafter referred to as ‘the system’) is an online platform that allows adult patients to contact their GP, access self-help information or learn more about NHS 111 (the UK NHS non-emergency number used to obtain advice for urgent healthcare needs) and local pharmacy services via their general practices’ website.24 To contact their GP, a patient completes an online form describing the nature of their problem (hereafter referred to as an ‘e-consultation’). This is submitted to their practice, which commits to responding by the end of the next working day. Thirty-six practices took part in the pilot and we use data from these to investigate when and how often the system was used. Patient-level data were obtained from eight practices to examine the types of users, the reasons for and the outcomes of e-consultations.
Data and data sources
Three data sources were used in this evaluation:
Routinely available data from Public Health England about general practices across England.25 The National General Practice Profiles provide data on each general practice that has a patient list size of at least 900 and was included in the 2014/15 quality and outcomes framework (QOF). Available data include local demography, QOF scores and GP patient satisfaction survey results.
Website analytics data provided by the software developers. These were summary data on the use of the website by each participating practice from April 2015 to June 2016. This included data on the number of page landings, number of unique users, number of e-consultations, the days and times e-consultations were performed, and the number of uses of the additional services (self-help, pharmacy locator and NHS 111 signposting).
Eight participating general practices were purposively sampled to ensure a range of locations (rural/suburban/urban), area deprivation using the English Index of Multiple Deprivation (IMD)26 and level of e-consultation use (high, medium and low, calculated by dividing the number of e-consultations by number of days the system was live). A random sample of patients who used an e-consultation were identified in each practice and anonymised data were abstracted from their electronic patient records (485 e-consultations equally distributed across the eight practices). Data were collected on patient demographics; reason for contact; nurse and doctor’s time and actions taken in response to the e-consultation (eg, telephone call, face-to-face appointment, request for tests and email advice); any further care provided by the practice in the 30 days following the e-consultation. A member of staff from one of the participating practices abstracted these data for all practices. An Excel spreadsheet was designed by the abstractor in conjunction with members of the research team (EM, WH, KN).
Data analysis
To examine the generalisability and potential selection bias in our results, we compared the 36 practices participating in the pilot to those in the rest of England. Any differences were assessed using Χ2 tests.
To determine how often and when the system was used, we present descriptive statistics (means and SD or absolute numbers and percentages) to show the number of patients using the system over time, the most popular days of the week and times of day (from both the website statistics and patient-level data).
For the eight practices providing patient-level data, demographic characteristics of users, reasons for consulting, actions taken in response to e-consultations, response times and durations of subsequent consultations were summarised using means and SD or absolute numbers and percentages. In addition, ORs with 95% CIs and p values were calculated. Reasons for consulting were categorised from a free-text field in the e-consultation into 11 broad categories (by HE at the time of analysis): musculoskeletal, infection/immunological, neurological, sexual/reproductive health, dermatological, respiratory, mental health, digestive, medication queries or advice, administrative (requests for fit notes (an official document regarding fitness to work), repeat prescriptions, test results, referrals and requests to book appointments) and unclear/not stated. The reason for an e-consultation was also cross-tabulated with the primary action taken to identify any patterns in types of response required. What happened as a result of the e-consultation in the subsequent 30 days is summarised as primary and secondary GP actions. The primary action is the most resource-intensive action taken by the GP and the secondary action is the next most resource intensive (the hierarchy was considered as the order of actions shown in table 4).
The time required to take the actions in response to an e-consultation by doctors and nurses, and any subsequent contacts within 30 days were abstracted from the individual patient records in the eight sampled practices described in data source three above. A triage time of 5 min per e-consultation was assumed based on responses obtained from interviews in the companion qualitative study where practice staff were asked to explain in detail how their practice dealt with e-consultations. Staff time was valued using 2015 unit costs for health and social care26 to derive the cost of e-consultations. This micro-costed estimate was compared with national published average costs for GP and nurse contact durations26 (see online supplementary table 1).
Supplemental material
Results
Study sample
Out of 102, 36 (35%) self-selected general practices within the study area took part in the pilot, covering a total patient population of 396 828. These 36 practices were broadly comparable to practices in the rest of England (table 1). The following differences were evident: participating practices tended to have larger average patient list sizes (11 023 vs 7321) and there were several markers of higher socioeconomic ratings in patients in participating practices compared with patients in the rest of England (more in full-time work/education (66.5% vs 60.8%), fewer with long-standing health conditions or caring responsibilities (50.6% vs 54.0% and 15.7% vs 18.1%, respectively) and fractionally higher life expectancy (84.0 vs 82.9 years). Study practices also tended to have slightly higher nurse and administrative staff support compared with the average practice in England.
Comparison of study practices with practices in the rest of England
Extent of interest in the system (web analytics)
There was a small amount of patient curiosity about the system, with a mean of 9.11 unique visits to the system website per 1000 patients per month (average minimum across practices: 1.41, average maximum across practices: 29.12 per 1000 patients per month). This included any time that someone looked at the website (‘page landings’), whether or not they went on to do an e-consultation or use the other services. The majority (71.1%) did not return to the website. Patients rarely clicked through to other information pages (the self-help link was viewed 3348 times, the pharmacy link 1744 times and the NHS 111 link 1527 times throughout the entire pilot, compared with 35 981 visits overall).
Extent of use of e-consultations (web analytics)
Website analytics data showed that the use of actual e-consultations was very low. Over the 36 participating practices, there was a mean of 2.00 e-consultations per 1000 patients per month (minimum 0.33, maximum 5.70 per 1000 patients per month). As the mean number of patients per practice was 11 023, this means that on average a practice might receive 18 e-consultations per month based on these figures. To put this into context, the most recent data on consultation rates in UK General Practice indicate that on average, there were 5.16 standard consultations (GP or nurse led) per patient per annum,1 equating to 4740 consultations per month in a practice with 11 023 patients. This means that e-consultations represent 0.002% of all consultations on average. There was a trend towards a gradual increase in use over the pilot period, although the last 3 months of the pilot saw a slight decrease in use (figures 1 and 2).
Total e-consultations over time, by day of the week (from website analytics data).
Total e-consultations over time, by time of day (from website analytics data).
Times of use (web analytics and patient-level data)
Patients were much more likely to use the system at the start of the week and during typical (UK) surgery opening hours. Of all e-consultations, 58.8% took place Monday to Wednesday, with use declining towards the end of the week: only 12.4% of e-consultations were completed at the weekend. However, 69.9% occurred between 7am and 5pm, with peaks seen around 10 am and 2 pm. There was some interest in evening use, although this was much lower than daytime use (19.4% between 7 pm and midnight, with a peak around 8 pm) (figures 1 and 2, table 2). When we looked just at the patient-level data (n=485 e-consultations), we showed that there were no differences in characteristics of daytime compared with evening users (p=0.715 for age, p=0.140 for gender, p=0.548 for pre-existing health conditions, p>0.08 for all categories of reason for e-consultation).
Use of e-consultations (from patient electronic medical records and e-consultation data)
User characteristics (patient-level data)
From the data abstracted from a random sample of e-consultations from eight practices it can be seen that women were almost twice as likely to perform an e-consultation as men (64.7% vs 35.3%) and over half of all users were age 25–44 years (median age 39, IQR 30–50). After the age of 45 years, use declined with age, although young adults (age 18–24 years) also accounted for only a small proportion of total usage (8.7%). The oldest patient using the system was 90 years old (table 2). There was no evidence to suggest that patient socioeconomic factors affected rates of use (p=0.755 for rates of use by IMD quartile, p=0.276 for employment status and p=0.696 for chronic health condition status).
Reasons for use (patient-level data)
Over a fifth of all e-consultations were for administrative reasons. The next most common reasons for an e-consultation were related to infections/immunological issues (14.4%) and musculoskeletal issues, such as back or knee pain (12.4%). In almost a third of cases, patients had consulted about the same issue within the previous 6 months, suggesting it was an ongoing health problem (table 3).
Reasons for e-consultations and practice responses (from patient-level data)
Practice response to e-consultations (patient-level data)
Practices had committed to responding by the end of the next working day after submission of an e-consultation. The median time to response was 1 day (IQR 1–3 days) and the maximum was 20 days, although response time of over a week was uncommon (table 3). E-consultations submitted on a Friday or Saturday predictably had a longer median time to response (3 days) compared with other days, as currently in the UK, GPs do not typically work on Saturdays or Sundays.
The most common primary response (defined as the most resource-intensive action) to an e-consultation was to arrange a face-to-face (38.1%) or telephone consultation (32.2%) with the patient. Other actions taken by the practice included issuing a prescription or fit note, requesting tests and giving advice. In about two-thirds of cases, a secondary (less resource-intensive) action was also taken, most commonly issuing a prescription or providing advice (table 3).
‘Administrative’ e-consultations were mainly dealt with via a telephone consultation (73.7%). About a half of all clinical (not administrative) e-consultations resulted in a face-to-face consultation (range, 39% for sexual health to 54% for neurological issues) and approximately a further third resulted in a telephone consultation (range, 21% for dermatological to 42% for infection/immunological issues).
After the initial response, a quarter of patients (25.8%) had a further consultation in the 30 days following their e-consultation. Of these, 19.5% were face-to-face and 5.6% were by telephone (table 3). Over half of the face-to-face consultations (57.5%) were with a GP, 17.0% with a practice nurse and 25.5% with another health professional (eg, healthcare assistant, practice pharmacist and phlebotomist). Almost all further telephone consultations (92.6%) were with a GP. Data on which health professionals had contact with the patients were only collected for follow-up actions, and not for the primary response (described in the preceding paragraph), but we could speculate that the distribution of staff resources there would be similar.
A face-to-face consultation was more likely to be needed for patients with a new condition compared with a pre-existing condition (OR 1.56, 95% CI 1.05 to 2.27, p=0.049, adjusted for age and gender). They were even more likely to be needed for patients who had not seen their GP about the problem in the preceding 30 days (OR 1.89, 95% CI 1.23 to 2.86, p=0.005).
Cost of e-consultations
The average duration of a face-to-face appointment in response to an e-consultation was 14.5 min and for a telephone consultation was 7.6 min. To put this into context, the most recently estimated national averages (2013–2014) are 9.2 min (95% CI 9.22 to 9.23) for face-to-face appointments and 5.4 min (95% CI 5.3 to 5.4) for telephone consultations.1 The average cost for the initial practice response to an e-consultation was £36.28. In context, the national estimates of cost for a standard GP face-to-face consultation is £33.00 (see online supplementary table 1). The cost was driven mainly by the time needed for a GP to triage the e-consultations and the relatively high proportion of e-consultations that resulted in a face-to-face or telephone consultation with a GP (table 4). When considering further follow-up actions taken in the subsequent 30 days, the average cost associated with an e-consultation increased to £45.39.
Average cost of all initial primary care actions in response to an e-consultation*
Discussion
Principal findings
We report on a 15-month pilot trial of an online consultation system in primary care. Principal findings from the website analytics data (36 practices) were that use of the system was very low, it was most likely to be used on weekdays, during typical UK surgery hours. From the patient-level data obtained from eight practices, we have shown that the system was used more commonly by women than men, and most often by working-age adults. The most common reason for an e-consultation was for an administrative request, for example, fit notes, repeat prescriptions and test results. The average time taken for a practice to respond to an e-consultation was 1 day, and about three-quarters of all e-consultations resulted in the patient being asked to arrange a face-to-face or telephone consultation. Patients reporting new conditions were more likely to require a face-to-face consultation than patients reporting on a pre-existing condition. The average cost to a general practice to respond to an e-consultation was £36.28.
Comparison with other studies
Our findings on the types of patients using the system, and the most popular days and times of use are consistent with findings from an earlier, smaller pilot study carried out by the software developers27 and from a second small study from the USA.28 However, when considering other types of technology, a survey across 14 European countries found that men were more likely to use email to communicate with their healthcare providers,15 while we found women were more likely to use online consultations. This discrepancy could be due to the difference in formats of email versus a structured online form but warrants further investigation in other populations. In regards to the reasons for consulting, the earlier pilot by the software developers27 found that mental and sexual health were two of the most common reasons for an e-consultation, whereas these were much less commonly cited in our study (5.9% and 8.5%, respectively). However, two randomised controlled trials (RCTs) of email systems/web-based online communication systems in primary care showed that both patients and clinicians felt that online/email communication would not be their preference or was less appropriate for mental and sexual health issues.29 30 While these RCT results cannot be directly compared with the earlier pilot study, there is a suggestion that these results are more in line with our own findings. Research in North America has reported that both patients10 and clinicians31 felt that email communication is particularly appropriate for administrative requests, which may help explain our observation that administrative requests were the most common reason for completing an e-consultation.
Possible explanations and implications
The very low use of e-consultations has several possible explanations. One is that few patients knew about the system, although all practices stated that they were advertising the system in various ways, though some were more active than others (banners in the practice, on their websites and answerphones, via texts to patients). Another explanation is that there is currently low demand for online consultations or at least in the format that they exist in the system trialled here. There are two key implications of this low usage. (1) Impact: part of the rationale for online consultations is that they may reduce staff workload and speed up patient access. However, if, as estimated here, e-consultations make up on average only a tiny proportion of all consultations (0.002%), the impact of introducing the system (without significantly increasing use) would be negligible. It is also not clear that use of the system would reduce, rather than increase staff workload (additional triage time and potentially longer resulting consultations). (2) Cost: for commercial online consultation systems operating on a per-patient cost model, much higher rates of use are likely to be necessary in order to make such systems cost-effective for practices. The systems are also expensive in terms of the work involved in using them and the potential duplication of effort in the cases where they lead to a standard consultation anyway. Further research is needed to improve our understanding of why e-consultation use is so low, and what could be done to increase it. Another issue for future research is that very low usage suggests that those who do use it may be atypical of primary care users. This possibility is supported by the observation that patterns in the age and gender profile of UK general practice users as reported in Hobbs et al 1 differ from the age and gender profile of e-consultation users as found here. It would be useful to elucidate more about the characteristics of those who choose to use e-consultations.
The low use of the system at weekends could simply reflect the fact that habits take time to change, and we are not yet used to the concept of being able to contact GPs at weekends. However, it might also indicate low demand for 7-day access to primary care services. The General Practice Patient Survey (covering >880 000 patients in England) found that >80% did not think they needed weekend (particularly Sunday) opening hours.32 Consistently with this, an evaluation of 7-day access pilots reported very low demand for weekend appointments (only 12% of all Sunday appointments filled),33 and some pilots have been discontinued due to lack of demand.34 Further follow-up and interviews with patients about their appetite for 7-day access are important to help understand this finding, particularly where policy-makers are committing to rolling-out 7-day access to primary care services.35
It might be anticipated that having already submitted a prior e-consultation, a follow-up face-to-face appointment about that issue would be shorter than at a standard consultation where the GP has no prior information about the issue. However, follow-up face-to-face appointments resulting from an e-consultation tended to be longer than the national averages (14.5 min compared with 9.2 min for face-to-face appointments, and 7.6 min compared with 5.4 min for telephone consultations).1 As this was not a controlled study, we were unable to directly compare consulting times with and without e-consultations, so this observation should be interpreted as suggestive only. The observation does however fit with the findings of a trial of preconsultation telephone triage in UK General Practices, where the prior telephone information from the patient made no difference to the duration of resulting face-to-face consultations.36 The implication is that collecting information in an e-consultation may not make subsequent consultations shorter, and it is possible that it increases, rather than reduces, clinical workload. This is particularly important if part of the rationale for such systems is to reduce the pressure on primary care staff. (We note that the software developers’ own earlier, smaller pilot study reported 10 min for follow-up face-to-face appointments, and 5 min for follow-up telephone calls,27 which are closer to the national average. However, it is not clear how these estimates were arrived at).
Online consultation systems could be amended to improve efficiency. For example, as booking a telephone consultation was the response to three-quarters of all queries regarding medications, it may be more efficient if the online system initially asked if patients had a medication query, and automatically directed them to book a telephone consultation if so. This would save time for clinicians processing an unnecessary form, and would speed up access for patients who otherwise spend time completing the full e-consultation, waiting for a response, and then needing to wait for a telephone consultation anyway. (We note that an ‘admin button’ has now been added to the system to deal with fit notes, test results and repeat prescriptions.) A second example is that of those who submitted an e-consultation; patients consulting about a pre-existing condition were far less likely to need a face-to-face consultation than patients consulting about a new condition. This could be because when a GP is already familiar with the patient and their condition, they may feel more confident proceeding without seeing the patient in person. This suggests that online consultation systems could potentially be more useful for patients consulting about a pre-existing condition. For new conditions that are likely to require a face-to-face appointment anyway, using the online system may unnecessarily add to delays and clinical workload by introducing another step in the process of getting an appointment.
Strengths and weaknesses
The key strength of this study is that it is, to our knowledge, the largest independent evaluation to date of a primary care online consultation system. A weakness is the observational nature of the study design that limits us to a descriptive evaluation. For the economic analyses we did not have data on which member of staff performed the triaging, as different practices processed their e-consultations in different ways. It was also not possible to know how much staff processing time was needed, we therefore had to assume triage times based on our qualitative findings (reported separately). It is therefore possible that the e-consultation cost is lower or higher than the £36 we have calculated, if triage is performed by lower grade staff or if processing time was greater than average. Future studies would benefit from quantifying this. It should also be noted that the figure of 5.16 consultations per patient per annum1 does not differentiate between consultations performed by GPs and those performed by nurses. This may impact our estimations.
Conclusions
Use of e-consultations was very low, particularly at weekends. Unless this can be improved, the impact of e-consultation systems on reducing staff workload and improving waiting times is likely to be negligible. It is also possible that use of e-consultations may be associated with increased costs and workload in primary care. Patterns in use suggest ways that online consultation systems could be developed to improve efficiency, such as channelling administrative requests separately and targeting the system specifically for patients who regularly see their GP for an ongoing condition. Future research should look at reasons for low uptake of e-consultations and try different approaches to improving uptake. If future research demonstrated significantly higher usage, this would pave the way for a feasibility study to more formally evaluate the effectiveness and cost-effectiveness of e-consultations.
Acknowledgments
Authors thank staff for participating practices and for assisting with the collection of individual-level data, and the software developers for providing web usage statistics.
References
Footnotes
Contributors KN, JH, HBE, MF, JB, EM, WH and EB: were responsible for the study design and collection of data. KN, JH and EB: were responsible for study management and co-ordination. CS: was a project advisor throughout. HBE, KN, EM and WH: analysed the data. HBE and KN: drafted the paper. All authors: read, commented on and approved the final manuscript.
Funding This research is part-funded by the One Care Consortium and supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care West at University Hospitals Bristol NHS Foundation Trust. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests EB is employed by One Care who part-funded the research. HBE, MF, EM, KN and JH are employed by CLAHRC West who received part-funding from One Care.
Ethics approval HRA (IRAS project ID: 204925; Protocol number: 2604; Sponsor: University of Bristol).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No further data are available.