Patient experiences of an online consultation system: a qualitative study in English primary care post-COVID-19

Background Online consultation systems (OCSs) allow patients to contact their healthcare teams online. Since 2020 they have been rapidly rolled out in primary care following policy initiatives and the COVID-19 pandemic. In-depth research of patients’ experiences using OCSs is lacking. Aim Explore patients’ experiences of using an OCS. Design and setting Qualitative study in English GP practices using the Patchs OCS (www.Patchs.ai) from March 2020 to July 2022. Method Thematic analysis of 25 patient interviews and 21 467 written comments from 11 851 patients who used the OCS from nine and 240 GP practices, respectively. Results Patients cited benefits of using the OCS as speed, flexibility, and efficiency. Nevertheless, some patients desired a return to traditional consultation methods. GP practices often did not clearly advertise the OCS or use it as patients expected, which caused frustration. Patients reported advantages of having a written record of consultations and the opportunity to communicate detailed queries in free text. Views differed on how the OCS influenced clinical safety and discussions of sensitive topics. Patients who struggled to communicate in traditional consultations often preferred using the OCS, and male patients reported being more likely to use it. Conclusion Globally, this is the largest in-depth study of patient experiences of an OCS. It contributes new knowledge that the patient experience of using OCSs can be influenced by previously unreported patient characteristics and the conditions they consult about. Further, it contributes recommendations on the design and implementation of the OCS in practice.


Introduction
Online consultation systems (OCSs) are digital communication tools that allow patients to request help from their healthcare team over the internet 1 by completing free text 2 or multiple-choice questionnaire forms. 3Practitioners can resolve patient queries by written message, telephone call, video consultation, or by arranging an in-person visit. 1,46][7][8] Their adoption was catalysed by the COVID-19 pandemic as a way for patients to receive care remotely and reduce the risk of spreading the virus. 9English GP practices have been mandated to offer OCSs since April 2020 as part of a 'Digital first primary care' policy. 10During the COVID-19 pandemic practices were advised to conduct 'Total digital triage' 11 where all initial patient contacts were made via an OCS.Since May 2020, OCSs have been available in 85% of English GP practices, 12 and, in those, using OCSs as their primary method of patient contact is estimated to account for 72% of all incoming patient requests. 13][16][17][18][19][20][21][22][23][24][25][26][27][28][29] This international research shows that OCSs can have positive impacts on care, such as increased access, though can have conflicting impacts on costs, workload, patient satisfaction, and care equity.
Research into experiences using OCSs has been largely conducted pre-COVID-19, and predominantly among clinicians. 17,18,30-32Exploration of patient views has been limited 33,34 with most studies employing survey-based methods, 24,28,[35][36][37][38] rather than qualitative interview-based techniques.Findings have often been hampered by low patient uptake or usage by specific patient groups such as younger patients. 2,32,39Given the increased adoption of OCSs since the COVID-19 pandemic, there is a need to explore patient experiences of using them in depth. 34

Online consultation system
Patchs functions in the same way as all OCSs: it is accessed online from the GP practice website and allows patients to request help from their GP practice in writing.Patients can submit clinical requests, for example, a new health problem or an ongoing health problem; and administrative requests, for example, a fit note or medication.A specific feature of Patchs is that patients describe their requests by providing unstructured free-text responses to open-ended questions in a 'chatbot', as opposed to filling in a multiple-choice questionnaire.It is estimated that approximately 13% of OCSs allow patients to solely describe their requests using free text. 34hatbot questions are fixed, though vary according to the type of request selected by patients.Questions cover topics of typical traditional primary care consultations, including a description of the request, and patients' ideas, concerns, and expectations. 40Staff aim to respond to patients within a stipulated timeframe set by the GP practicefor example, 48 working hours, either by written message, video/telephone call, or by arranging an in-person visit.Supplementary Box S1 provides a full description of the system using the Template for intervention description and replication (TIDieR) checklist. 41is study aimed to explore patients' experiences using OCSs since the COVID-19 pandemic.Through an existing research collaboration, this study focused on one OCS called Patchs (Patchs Health, London; www.Patchs.ai),which has been available to GP practices in England since December 2019.Specifically, the researchers sought to explore why patients chose to use the OCS, the circumstances in which they preferred to use it, and the perceived facilitators and barriers to its use.The authors anticipate the findings will be relevant to other OCSs therefore adding value for all GP practices using OCSs and other OCS designers.

Method
The researchers conducted a qualitative study using semi-structured interviews triangulated with large volumes of written feedback to explore patients' experiences of using the specified OCS named Patchs.

Setting and sample
A sample of GP practices (n = 240) were included in the study from March 2020 to July 2022: London (n = 153), Southeast England (n = 12), North West England (n = 44), South England (n = 25), Midlands (n = 4), and Wales (n = 2).All GP practices that used the OCS during the study period were included.

Data collection
A total of 25 semi-structured interviews were conducted by telephone by either of the first three authors.All patients submitting feedback, both positive and negative, were asked if they consented to being contacted by a member of the research team to take part in an interview about their experience.Interviewees were recruited via email; sampling was purposive and maximum variation was sought for factors that could influence OCS experience including age, sex, and ethnicity 34 (Table 1).The interview guide explored the strengths, weaknesses, opportunities, and threats of OCSs in general, such as online access and written communication, using Patchs as an example (Supplementary Box S2).Interviews were audio-recorded and transcribed verbatim.
Written free-text feedback was obtained from all 240 practices and included 21 467 comments from 11 851 patients.Feedback was collected in the OCS using the question: 'How was your experience using Patchs today?' in a free-text box of unlimited character count ('Please tell us more in the box below').Patients were prompted to provide feedback after each interaction with the OCS -for example when submitting a request, replying to a message, or on completion of a consultation.Patients could also provide feedback at any time by accessing a 'Give feedback' link in the OCS.Supplementary Box S3 provides further information on data collection.

Data analysis
Interview transcripts and written feedback comments were imported into NVivo (version 12).Using thematic analysis, 42 codes were inductively developed and transcripts independently coded line by line by at least two out of three primary authors.Findings were triangulated between interviews and written comments using a convergence coding matrix that displayed data from both sources enabling data to be compared and contrasted. 43Data analysis was conducted alongside data collection; both ceased at saturation when themes were fully developed with clear definitions, and no new information emerged after at least three interviews. 42upplementary Box S3 provides further information on data analysis.

Results
The GP practices (n = 240) had used the OCS for a range of 1 to 31 months by the end of the study.

How this fits in
Online consultation systems (OCSs) have been rolled out rapidly, but little is known about patients' experiences when using them.This study was the largest ever reported qualitative study of patient experiences using an OCS.The findings provide insight into why some patients prefer in-person consultations, and why others prefer to use an OCS.Patients' experiences of using OCSs can be influenced by how they are designed, how GP practices use them, patient characteristics, and the type of request.In total 230 patients were invited to participate and interviews were conducted on all those who responded (n = 25; 11%).The semi-structured interviews that were conducted with 25 patients had an average duration of 38.5 minutes (range 16 to 62 minutes) and were from nine GP practices who had used the OCS for 14.4 months on average (range 6 to 20 months; Supplementary Table S1) at time of interviews.
Findings from interviews and written feedback were similar and are reported together with example quotes from both data sources.Additional quotes are shown in Supplementary Table S2.

Perceived benefits of the online consultation system
Most patients identified advantages of the OCS over traditional methods of contacting their GP practice, especially for 'simple' health problems such as 'rashes' or 'colds'.The primary benefit reported by most patients was the ability to receive a quick response to their query, compared with the delay (typically 2-3 weeks) associated with traditional routine inperson appointments.The convenience and flexibility of the OCS in terms of how and when it could be accessed, particularly for patients who worked full time or had childcare considerations, was also advantageous.Patients believed the OCS saved them time by avoiding busy phone lines to reception, unnecessary travel, and long waits due to delayed appointments: 'The advantages to me were it was fairly quick, there was less wait than there would be if I had to go to the surgery, or had to make an appointment.It was more instantaneous.'(Interview, practice [PR] 2, patient [P] 2, male [M ], aged 74 years) The ability to provide their GP with detailed information in advance, should a telephone or in-person consultation be necessary, was also felt to be efficient: 'I mean, it saves me time, as well, because like everybody, I'm time poor, so being able to do this online and then have a chat with the doctor when they phone me back, and I've already given them all the details they need -I think it works very well for me.I think it's brilliant.'(Interview, Pr1, P3, M, aged 61 years) Despite the perceived benefits of the OCS, patients often had negative experiences.However, these were largely related to how the GP practice implemented and used the OCS, rather than the design of the OCS itself.This is explored in detail in the next section.

GP practice implementation of the online consultation system
Communication about the service.Poor communication about the OCS often left patients disappointed and frustrated.Some practices adopted a 'soft launch' where the OCS was not available to all patients initially owing to fear of being inundated.When made more widely available, patients were often not notified in advance, and only learned about the OCS from reception staff at the point they were unwell and needed help.GP practice websites were frequently criticised for being overloaded with information, and needed clearer signposting to the OCS.Some patients were confused about the purpose of the OCS, particularly when their practice offered other online patient-facing systems with different functions, for example, to order prescriptions.Some patients were unaware that in-person appointments would be offered if necessary, whereas others thought the OCS was an in-person appointment booking service: Expectation versus reality.Patients expected certain events to happen after submitting a request.If it did not happen it negatively impacted their experience, which led to mistrust and reluctance to use the OCS again.Patients expected to receive acknowledgement from the practice of their OCS request soon after submission, and for it to be fully resolved within the timeframe they advertised, for example, 48 working hours.If this did not happen, they often telephoned the GP practice to chase a response.Patients expressed irritation when the GP had not read the details of their request before an in-person or telephone consultation, which dissuaded them from providing in-depth information in future:

'The problem on both occasions hasn't been the logging of information from my side. It's what happens with that information when it goes to the GP practice … Neither of them [GPs] accessed the notes I'd put on [the OCS]. They had no idea what the consultation was going to be about … It's put me off putting any detail in.' (Interview, Pr7, P22, F, aged 45 years)
Patients expected their request to be actioned appropriately and informed of how their request was resolved.They also wanted the opportunity to ask follow-up questions if anything was unclear: 'In this instance the case has just been closed off by the reception team, however the feedback I have does not fully help me so now I can either raise another request explaining the whole matter again or call the surgery which is what I thought the site was trying to prevent.' (Feedback, Pr5, F, aged 38 years) A small number of feedback patients preferred that a particular doctor resolve their request.However, all interviewees reported they did not see the same GP before their practice's implementation of the OCS anyway.Most patients reported that they valued speed of response over continuity of care: 'Perhaps a question could be, do you want the request dealt with by the same doctor or by the one who can deal with it the quickest?' (Feedback, Pr53, M, aged 75 years) Access management.The way practices managed patient demand through the OCS varied significantly, and was a prominent theme in the feedback data.Many GP practices limited the availability and number of requests that could be submitted by patients each day to avoid being overwhelmed.Uptake of the OCS was high and demand almost always exceeded these limits.This meant patients could often not access the OCS when they wanted to, thus removing its benefit of flexibility: 'It excludes people who have jobs and responsibilities as not everyone has access to the internet at seven thirty in the morning.' (Feedback, Pr18, F, aged 23 years) However, some GP practices limited availability of the OCS with minimal impact on patient experience.These practices were clear about when the OCS was available, had staggered opening times throughout the day, had higher limits on request numbers, and allowed administrative request submissions when closed for clinical queries.
'Personal' service.Many patients reported the OCS had a positive impact on the relationship with their GP practice.Key to positive experiences was the perception of a 'personal' service, which included staff acknowledging patient concerns, swift replies to requests, and direct contact from a GP -either by written message or telephone: 'Even the doctor herself got involved in the delivery of the advice and the guidance.Either by picking up the phone directly or by … messaging through the system [it] worked very well.' (Interview, Pr2, P6, M, aged 58 years) Though the OCS was embraced at the start of the pandemic, feedback comments showed some patients believed they were a temporary response to COVID-19.These patients were frustrated and expected default in-person consultations to return: 'I am trying to reach out to my GP but these online services are making it impossible … these GPs are now hiding behind their computers and patients are being ignored and made to feel helpless.'(Feedback, Pr61, F, aged 62 years) These patients found the OCS impersonal and felt detached from their GP practice.This was particularly evident in practices using 'Total digital triage ', 11 where patients often felt forced into using the OCS.They were disappointed they could no longer chat to reception staff freely and felt that in-person appointments would become increasingly rare: 'I'm concerned there could be a tendency for it to be so successful from their side that face-to-face appointments be less of an option.' (Interview, Pr6, P18, F, aged 57 years) Support and guidance.Though most patients said the OCS was easy to use, those lacking confidence with technology in general felt empowered when they received support by practice staff.

Written communication between patient and GP practice
Written record.Almost all patients felt it was beneficial to have a written record of communication with their GP practice.They often found it difficult to remember information given verbally in traditional consultations, and liked the ability to refer back later: 'I prefer it, because it's written down again.Verbally you might not take it all in at the time.So, you've got that physical hard copy to keep.' (Interview, Pr2, P9, M, aged 39 years) Opportunity to communicate detailed queries in free text.Though some patients found the chatbot questions impersonal, most thought they were useful prompts that mirrored conversations with clinicians in traditional consultations.Patients liked the ability to respond to the questions using unstructured free text and describe concerns in their own words.Some patients had used other OCSs that used multiple-choice questionnaires, which they found to be restrictive, complicated, and time-consuming: 'This [OCS] so far looks very good as it is easier and more efficient than [previous multiple-choice questionnaire system].
[Previous system] asked many unnecessary questions that had no relevance to the consultation, but this is direct and easy to use asking specific questions.' (Feedback, Pr21, M, aged 34 years) Patients liked the ability to write, review, and edit queries in their own time.They compared the OCS to traditional in-person appointments, which were time limited.Patients often felt rushed during traditional consultations and worried they would forget to share key information.Others felt the OCS questions compelled them to provide more detailed information: A minority of patients feared that using the OCS may misdirect GPs to an incorrect diagnosis.They believed the OCS placed responsibility on them to share key information in sufficient detail.
Others worried the lack of non-verbal cues and wider physical context of in-person appointments could result in serious issues being overlooked: 'That would be the danger I would feel, that the doctor makes an assessment presumably how the doctor sees you.Does she see you as well in general or does she spot something else that's maybe wrong.You go in with your big toe and she may think, mmm, something doesn't look quite right there in some other area.' (Interview, Pr5, P16, M, aged 73 years) Preference for online consultation system use according to patient and request characteristics Sensitive topics.Some patients felt in-person consultations were most appropriate for 'sensitive' topics such as mental and sexual health issues.They believed the pre-set chatbot questions were impersonal and uncaring, and that patients may be forced to articulate emotional topics twice -initially via the OCS, and then again verbally in a telephone or in-person consultation: 'I think part of the benefit of face-to-face is that you have the conversations and quite often they bring out other aspects.So with [the OCS] I think you might find it quite difficult to deal with issues like mental health because it's quite hard to explain in those boxes and within those questions …' (Interview, Pr5, P13, F, aged 40 years) In contrast, some patients felt anxious when discussing sensitive topics on the telephone and in-person, and preferred to use the OCS precisely because it was impersonal.When using the OCS they felt detached and less embarrassed, meaning they could communicate more openly: 'You know, you might be embarrassed or something like that, with mental health issues and stuff like that, you might not necessarily work good seeing someone face-to-face, and you can actually get your point across a bit better writing it down.'(Interview, Pr7, P21, M, aged 35 years)

Box 1. Recommendations for GP practices using online consultation systems to optimise patient experience
Clearly communicate to patients about the service Let them know when an online consultation system (OCS) is launched, and how it can be accessed.OCSs are typically accessed via the GP practice website, so they should be prominent and easy to find.Be clear what the OCS should be used for, including whether in-person appointments will be arranged if necessary -particularly if other online services are also available.

Practise good OCS 'etiquette'
Send a brief acknowledgement message to patients to confirm the practice has received their request as soon as possible, and then contact patients within the stated timeframe.Read the request first -do not ask patients to repeat what they have already written.Action the request and inform the patient how their request was resolved.Offer the opportunity for patients to ask questions.This will encourage patients to provide detail in future requests, and avoid them telephoning the practice to chase responses and submitting duplicate queries.

Manage OCS access fairly
If availability and number of OCS requests patients can submit is limited, communicate clearly to patients when OCSs will be available, and stagger opening times throughout the day, for example, not just at 8:00 am.Maximise the number of queries the practice will accept, and allow administrative queries to be submitted when closed for clinical ones -for example, requests for fit notes and repeat prescriptions.

Support patients who struggle to use OCSs
By providing support on the telephone if needed.

Provide a 'personal' service
Strengthen the relationship with patients by acknowledging their concerns, replying quickly to their queries, facilitating direct contact with a GP where appropriate, for example, by phone or written message and arranging in-person appointments if preferred.

Make other options available
Some patients will not want to use an OCS to discuss specific conditions -for example, sensitive topics -and some may struggle to use them because of preexisting diagnoses -for example, attention deficit hyperactivity disorder.Some may not have a clear reason for not wanting to use OCSs.Other options to access the GP practice, such as telephone or in-person appointments, should be available.
Promote OCSs to patients who may traditionally struggle to access primary care Some patient groups who traditionally struggle to access primary care may derive specific advantages from using OCSs, such as those with hearing loss, anxiety, and autistic spectrum disorder.

Do not make assumptions about who may not want to use an OCS
Certain patients may counterintuitively prefer to use OCSs, such as older people and those who speak English as a second language.Enable patients to use OCSs when they would prefer.

Research
Patients with communication challenges.Patients who struggled with verbal communication in traditional telephone or in-person appointments usually found it easier to use the OCS.This included patients with hearing loss, anxiety, and autistic spectrum disorders.Patients for whom English was a second language often preferred the OCS because they were more confident writing rather than speaking English, and had access to free online tools to support them:

'If I was face to face, I would probably have to do a lot of paraphrasing … having the doctor guessing sometimes what I mean. Whereas now that it's typing I can just use, I don't know, the [online automatic translation service] or the [health website]
for similar symptoms and see what, not particularly the medical term but maybe the more or less slang term, if that makes sense.' (Interview, Pr2, P7, F, aged 38 years) In contrast, feedback indicated that some patients with attention deficit hyperactivity disorder (ADHD) found the OCS difficult to use owing to the written nature of the communication:

'I have ADHD … I also do not have the time or patience and often even skill to sign into
[OCS] to retrieve messages … I don't want to have to type my flipping issues when I am distressed and needing to speak with a doctor.' (Feedback, Pr68, F with ADHD, aged 41 years) Male patients.Male patients reported they were more likely to contact their GP practice using the OCS.Some were concerned they were wasting GP time and believed OCSs were more efficient for practices.Others admitted to delaying contacting their GP practice as an avoidance tactic to cope with anxiety about an underlying serious diagnosis.The impersonal nature of the OCS, and the act of writing down their concerns, helped them emotionally detach:

'I'm always reluctant to go to the doctor, I almost hope things will clear up if I leave it a week or so … I'm slightly distanced here, I'm a bit worried about this and maybe there's something wrong … I can do that a little removed and it cushions me from having actually to ring the surgery or go
to the surgery and say, oh, I'm worried.' (Interview, Pr5, P16, M, aged 73 years) Older patients.A common preconception was that OCSs would exclude older patients.Though initially apprehensive about the OCS, many older participants found the system easier to navigate than expected and often preferred using it to contact their GP practice than traditional methods.

Summary
This qualitative in-depth study explored the experience of patients using an OCS in English primary care during and after the peak of the COVID-19 pandemic when uptake was high, and is the first and largest of its kind.
Overall, patients had positive experiences using the OCS, though this was influenced by how GP practices used them, the design of the technology, characteristics of the patient, and clinical condition they used the OCS for.In general, patients believed the OCS was quicker, more flexible, and more efficient than traditional consulting methods.Despite this, some still wanted a return to traditional ways of accessing their GP practice via telephone and in-person visits.There was substantial variation in how GP practices implemented and used the OCS, which was often the key driver of patient experience.Most patients welcomed the ability to communicate with their GP practice in free-text writing using the OCS, though views differed on whether this had positive or negative impacts on clinical safety.Some patients preferred discussing sensitive topics using the OCS, whereas others preferred in-person consultations.Patients who struggled to communicate verbally in traditional GP consultations found it easier to use the OCS, whereas those with attention disorders found it more difficult.Male patients reported being more likely to contact their GP practice using the OCS, and many older patients found the system easy to use.Further research is needed to identify which patients did not use the OCS and why.
This study provides new insights into why some patients may prefer to use OCSs or traditional ways of contacting their GP practice.GP practices should focus on clear communication about OCSs, good practice OCS 'etiquette', and manage OCS availability fairly.

Strengths and limitations
This study generated rich narratives from patients by triangulating qualitative data from in-depth interviews and large volumes of written feedback.Globally, qualitative research studies exploring patient experiences of using OCSs before the COVID-19 pandemic were often limited by low levels of adoption or usage by specific patient groups, such as younger patients. 2,32,37,39,44This study was conducted after the start of the COVID-19 pandemic when GP practices implemented the OCS into their daily workflows and saw high patient uptake.Patients in the present study had typically used the OCS multiple times and came with a broad range of characteristics, including ethnic minority backgrounds, patients who speak English as a second language, and older people, thus providing insights into factors affecting high adoption that have not been previously reported, 2,32,39 such as access management and evolving attitudes towards OCSs over time.

Box 2. Recommendations for online consultation system designers to optimise patient experience
Allow patients to write their queries using free text Patients who had previously used multiple-choice questionnaires reported them to be restrictive and time consuming.This study found that patients liked the ability to openly describe their problems in detail using their own words.

Allow patients to ask follow-up questions
A common complaint was the perception that the patient's request was closed prematurely.This resulted in a reluctance to trust the OCS in the future.In this study, patients frequently requested the ability to clarify responses from their GP practice and to ask further questions before their request was closed.

Manage expectations
Patients often reported confusion as to the purpose of how the GP practice was using the OCS and were then disappointed if it did not meet their expectations.OCS providers should explain clearly to patients what OCSs should (and should not) be used for, how their queries may be resolved by their GP practice -for example, by arranging in-person appointments -and how to get help if OCSs are temporarily unavailable.
Though only a small proportion of participants who were approached agreed to be interviewed (11%), participants had varied experiences of the OCS.Patients who left feedback comments had both positive and negative experiences (Supplementary Box S3), therefore inviting them to be interviewed after leaving feedback could be viewed as a strength of the recruitment method.
A potential limitation of this study was that patient experience was evaluated using only one OCS.To counteract this the researchers focused on reporting findings related to functions shared by all OCSs, such as online access, and written communication between patients and their healthcare teams, rather than features unique to Patchs.Additionally, some participants had used other OCSs and were able to compare and contrast their experiences of using different features, such as describing their requests as free text versus multiplechoice questionnaires.A further potential limitation was that the experiences of patients who did not use the OCS was not explored.This may have provided further insights into barriers to adopting the OCS, and its broader impacts on patient care.

Comparisons with existing literature
Consistent with pre-COVID-19 research conducted internationally, the authors found overall high patient satisfaction with OCSs 2,3,15,21,28,36,37,39, [45][46][47][48] and varying views on their safety. 14,18,21,27,28,44,47,49owever, they also found a subgroup of patients with an evolving negative attitude towards their ongoing use after the pandemic peak.These patients wanted a return to traditional inperson appointments by default, citing concerns about clinical safety and a lack of personal connection to their GP practice.Some, however, could not articulate specific reasons why in-person consultations were preferable.These views may have been fuelled by rhetoric in the British media and the government that GP practices were not conducting in-person appointments during the COVID-19 pandemic. 50,51ior qualitative research has identified advantages for specific groups of patients when using OCSs, including those with hearing loss, 2 anxiety, 38 work commitments, or care responsibilities, 35 all of which are corroborated by the present findings.This study found new evidence that male patients and those with autistic spectrum disorder may also benefit from using OCSs, though the latter will likely only benefit if the condition does not also impact written communication skills. 52This study further identified that patients with ADHD are potentially a new group of patients that may find using OCSs difficult.
Previous qualitative research has also shown that GP practice staff have concerns that older patients 2 and those who speak English as a second language 35 may find OCSs difficult to use.This appears to be supported by quantitative research that shows these patient groups use OCSs less frequently than younger patients and native English language speakers. 48The results from the study presented in this article provide new insights that these groups can still derive benefits from OCSs, and that some may actually prefer using them to traditional consultation methods.

Implications for research and practice
This study found that how GP practices used the OCS and aspects of its design were key drivers of patient experience.Drawing directly from the present findings, the authors make specific recommendations for GP practices and OCS designers on how they can optimise patients' experience when using OCSs (Box 1 and Box 2, respectively).NHS guidance recommends that free online translation tools should not be used in clinical practice because their accuracy cannot be verified. 53However, the present results show that patients will often use them without the knowledge of their GP practice; further consideration for how this could be managed by GP practices and OCS designers is required.

Table 1 Participant characteristics Characteristic Patients interviewed (n = 25) Patient feedback contributors (n = 11 851)
a Groups are not mutually exclusive so do not add up to 100%.IQR = interquartile range.OCS = online consultation system.SD = standard deviation.
. Cowie J, Calveley E, Bowers G, Bowers J. Evaluation of a digital consultation and self-care advice tool in primary care: a multi-methods study.Int J Environ Res Public Health 2018; 15(5): 896.29.Wilson G, Currie O, Bidwell S, et al.Empty waiting rooms: the New Zealand general practice experience with telehealth during the COVID-19 pandemic.N Z Med J 2021; Farr M, Edwards H, et al.Use of an electronic consultation system in primary care: a qualitative interview study.Br J Gen Pract 2018; DOI: https://doi.org/10.3399/bjgp17X693509.32.Casey M, Shaw S, Swinglehurst D. Experiences with online consultation systems in primary care: case study of one early adopter site.Br J Gen Pract 2017; DOI: https://doi.org/10.3399/bjgp17X693137.33.Mold F, Hendy J, Lai Y-L, de Lusignan S. Electronic consultation in primary care between providers and patients: systematic review.JMIR Med Inform 2019; 7(4): e13042.34.Darley S, Coulson T, Peek N, et al.Understanding how the design and implementation of online consultations affect primary care quality: systematic review of evidence with recommendations for designers, providers, and researchers.