METHOD
General practices in south-west England and Cambridgeshire were eligible to participate in the exploratory trial if their score for communication items had been in the lowest 50% in the previous year’s (2013) National GP Patient Survey.9 Reasonable travelling distance from the two research centres guided selection within the sampling frame. Practices were invited to participate in the exploratory trial until the target (10) was reached. Sampling was undertaken in this way to target an area of clinical and professional practice — communication — that is important to patients10 and may be amenable to change.11
Real-time feedback kiosks were installed in the waiting areas of 10 practices (n = 8 south-west England, n = 2 Cambridgeshire) for 12 weeks. During this period, all patients attending the practice were invited to provide feedback about the care they had received. Survey items are detailed in Box 1. Practices were sent fortnightly reports based on this feedback.
Box 1. Real-time feedback survey items
| Item wording | Response options presented |
|---|
| • How likely are you to recommend our GP surgery to friends and family? | Extremely likely/Likely/Neither likely nor unlikely/Unlikely/Extremely unlikely /Don’t know |
| • How easy is it to get through on the telephone to this practice? | Very easy/Fairly easy/Not very easy/Not at all easy/Haven’t tried or Don’t know |
| • How easy is it to get an appointment for a time that suits you? | Very easy/Fairly easy/Not very easy/Not at all easy/Haven’t tried or Don’t know |
| • How helpful do you find the receptionists at this GP surgery or health centre? | Very helpful/Fairly helpful/Not very helpful/Not at all helpful/Don’t know |
| • Overall, how satisfied are you with the care you get at this GP surgery or health centre? | Very satisfied/Fairly satisfied/Neither satisfied nor dissatisfied/Fairly dissatisfied/Very dissatisfied |
| • Have you had an appointment with a health professional at the practice today? | Yes/No |
• If ‘Yes’:
Which of the following health professionals did you see? | Doctor/Nurse/Healthcare assistant or phlebotomist (for a blood test)/Practice counsellor/Other health professional |
| • Which doctor or nurse did you see today? | List (and photographs) of individual staff at the practice plus: Another doctor/Another nurse/Don’t know |
| • Do you have confidence and trust in the doctor or nurse you saw today? | Yes, definitely/Yes, to some extent/No, not at all/Don’t know or Can’t say |
• How good was the health professional at each of the following …
Giving you enough time Listening to you Treating you with care and concern Taking your problems seriously
| Very good/Good/Neither good nor poor/Poor/Very poor/Doesn’t apply |
| Up to two items (with relevant response options) on topics selected by the practice team were included after the clinician communication skills items, or (for patients who had not consulted a health professional) after the overall experience item. |
| • Are you …? | The patient/Parent or guardian of the patient/Spouse or partner of the patient/Another relative or friend of the patient/Other |
| • Are you/Is the patient …? | Male/Female |
| • How old are you/is the patient? | <18/18–25 years/26–45 years/46–65 years/>65 years |
| • What is your/the patient’s ethnic group? | White/Mixed/Asian or Asian British/Black or black British/Chinese or Other |
| • If you would like to leave any further comments, please type below | Space for free-text comments |
Following the real-time feedback implementation period, and depending on practice preference, researchers conducted either semi-structured interviews with a purposive sample of staff, ensuring all professions were represented, or a practice-based focus group to which all staff were invited. These sessions explored aspects of the real-time feedback implementation including training, technical support, using real-time feedback to collect patients’ views, the content of the fortnightly reports, and the impact of receiving feedback. Interviews (lasting 20–30 minutes) and focus groups (lasting 40–45 minutes) were audiorecorded and transcribed verbatim.
How this fits in
Although it is possible to implement real-time feedback successfully in most GP practices, its potential for contributing to service improvement may not be realised unless the facilitators and barriers to full practice engagement are understood and addressed. This study highlights the potential of real-time feedback for contributing to improved patient experience. It emphasises the need for effective communication within general practices to ensure that the processes involved in obtaining, and acting on, patient feedback are understood by all staff, enabling patient concerns to be promptly and effectively addressed.
QSR NVivo 10© was used for coding, linking, and retrieving the qualitative data from the sources described above. After initially reading a sample of the transcripts independently, two qualitative researchers noted and discussed preliminary themes. Any differences were resolved, and duplications were eliminated.
Normalisation process theory (NPT),12 originally developed to understand the embedding of new technologies into health systems, guided the production of a coding framework for the refined themes. NPT is underpinned by four constructs:
coherence (how people make sense of a new system/process);
cognitive participation (how people engage with it);
collective action (how people make new systems work in practice); and
reflexive monitoring (how people assess new systems’ value).
According to NPT, these constructs often operate and are experienced simultaneously.12,13 Two researchers used the NPT framework to analyse and code the transcripts, including any data that did not appear to fit neatly. The analysis was discussed in regular sessions with a third researcher, who managed qualitative and quantitative aspects of the real-time feedback research. Data identified as falling outside the NPT constructs concerned details of the research process, rather than the findings of this specific study, so have not been explored for the purposes of this article.
RESULTS
After initially approaching a total of 28 practices, 10 were recruited. Nine of the 28 practices declined to participate and nine had not responded by the time the target was achieved. Participating practices were located in a variety of settings, with a range of list sizes and deprivation deciles. Staff from four practices took part in focus groups. Interviews were conducted with staff members (n = 22) from the remaining six practices. Table 1 details practice and staff characteristics.
Table 1. Characteristics of participating practices and staff
The analyses of interviews and focus group discussions suggested that a range of cultures and communication styles existed within participating practices, although no formal assessment of these was undertaken.
Engaging with real-time feedback and making it work was notably more successful in practices with an open, inclusive communication style. These practices tended to include all staff members in the real-time feedback implementation and discuss feedback as a multidisciplinary group.
In practices with a less inclusive culture, real-time feedback was sometimes viewed with suspicion or ignored altogether; in these practices, staff were inclined to work predominantly within their own professional group and all-practice discussions appeared to be rare.
Making sense of the real-time feedback implementation
Most practices were used to obtaining and handling feedback from their patients, but real-time feedback was new to staff involved in this study; they made sense of it by comparing it with more familiar feedback methods.
Some favoured the convenience and immediacy of real-time feedback when comparing it with traditional paper-based surveys, which often entailed additional work for the practice, including data analysis:
‘With forms they were … maybe taken away and told to bring them back later and perhaps never did, whereas this was like, well you’re doing it now, and it’s done.’
(069005, deputy practice manager, interview)
However, by contrast, others highlighted problems with feedback from patients who had not had time to reflect on their experience:
‘You’re gonna get some hot-headed responses aren’t you? I mean people are going to come out and get really cross, there’ll be some emotion going on there that if they cooled down for 5 minutes you wouldn’t get those responses.’
(015009, practice nurse, focus group)
For some practices, real-time feedback was part of a strategy for obtaining patient feedback, and valued as an additional means of staying abreast of patients’ concerns.
Some staff mentioned links between real-time feedback and other schemes, such as the Friends and Family Test (FFT)15 and GMC appraisals,16 and viewed the extra data provided by real-time feedback positively:
‘... we’re very familiar with GP surveys … I think everybody was very enthusiastic that it was an alternative form of this, rather than the paper ones that we always struggled with … so having an alternative which is sort of there and part of the furniture and whatever was actually really quite nice.’
(017019, practice manager, interview)
Staff drew on personal experience of customer service initiatives in other contexts, recognising that nowadays people are frequently asked for feedback about a range of services, not only health care. These experiences made real-time feedback feel familiar.
Engaging with real-time feedback
Findings about how practices engaged with real-time feedback, and made it work in practice, are presented together. These were considered through the lens of two related NPT constructs — cognitive participation and collective action — which focus on how people interact with, and work to embed, an innovation.
Staff relationships with each other and with patients were a crucial part of the real-time feedback implementation. The level of communication concerning feedback from patients within practices varied among the participating practices:
‘There’s nothing that’s kept away from us … whether it be good or bad … if we have to do something to either make it better or keep up what we’re doing then they [patients] tell us.’
(007021, administrative staff, interview)
‘We do get some feedback but it’s not a lot. I think it’s almost like an e-mail to say if you want to look at it you can look at it on there, you know … we never have a meeting where we are all in the room and discussing it, we don’t discuss anything.’
(069013, receptionist, interview)
Individuals within the same staff team in a practice also had differing communication styles. Some reception staff had sufficient confidence to encourage patients to leave real-time feedback, but others did not.
Similarly, some GPs were reluctant to ask their patients for feedback. One suggested that requesting feedback tarnished an otherwise positive consultation:
‘It can feel awkward … if the conversation has gone really well, it sometimes slightly undermines the goodness of the conversation or the help that you’ve given.’
(007002, GP, interview)
Timing of the request for feedback was an important consideration. Several interviewees mentioned the problem of asking patients to leave feedback after a consultation when the patient has little time, or is busy with responsibilities for children or older relatives. Some thought that patients may have more time before a consultation, while in the practice and waiting to be seen:
‘It’s much easier to get patients to fill things in while they’re waiting than when they’ve finished ... because they’re sitting down waiting … and looking for something to do.’
(011018, office manager)
Staff from one practice noticed that patients attending a one-off influenza clinic were particularly willing to leave feedback. Although sometimes practices viewed concurrent requests for other forms of feedback (such as questionnaires or the FFT) as problematic, real-time feedback was mainly viewed as part of a programme of practice–patient communication, rather than a competing initiative. Generally, however, staff said they could not prioritise requests for feedback at busy times:
‘When we’re queuing three people … it’s the last thing on our minds, is to say “oh, can you go and use the feedback machine?”, to be honest with you.’
(017025, receptionist, interview)
Appraising and learning
Staff expressed a variety of views about the developmental value of real-time feedback to themselves as individuals and for their practice team. Many commented on how real-time feedback complemented, or duplicated, other forms of feedback; sometimes staff were happy to have confirmation of messages received via other feedback processes, but sometimes the duplication was used to disparage real-time feedback as a learning tool:
‘… we had our CQC inspection, and we had our report, around the same time as we were doing this, so it kind of all fitted in quite nicely together, and then we’ve had our patient participation questionnaire … I think it’s also useful to get feedback in different ways, and not rely too much on one method, ‘cos some of them can give quite different pictures or they can confirm the kind of whole.’
(011017, practice manager)
‘The trouble is a lot of what they tell us are things that we already know and the trouble is because of the way the system runs whether it be the structure of the building or whether it be the constraints that the NHS puts on me, y’know, the fact that I haven’t got a million pounds to spend today, it’s essentially I do what I can with what I’ve got and what I’m given and so therefore can’t, I think most of what they tell me I already know. Or if I don’t know it’s probably not something I can easily change.’
(021001, GP)
Many staff noticed low real-time feedback completion rates, and feared that the majority of patients had not been given, or taken, the chance to participate. Some believed that real-time feedback attracted only patients with extreme views, excluding the ‘middle ground’; the immediacy of the feedback (just after a consultation) was seen as encouraging patients with an axe to grind and excluding those with more moderate views. For these reasons, the rationale for acting on real-time feedback was sometimes felt to be quite limited:
‘It attracts two types of people … the people who love you and tell you they love you and the people that just had a really bad experience that day and want to take it out on the system really.’
(015016, practice manager, focus group)
Some staff were concerned that real-time feedback may exclude the views of patients with limited language or literacy skills (including some ethnic minorities, migrants, refugees, and asylum seekers); likewise, some suggested that older people may be uncomfortable using new technology to give feedback.
Several commented on the content of the real-time feedback survey, and criticised the number of demographic questions included; being generally positive about tailoring the survey to make it relevant to their organisation, they indicated they would appreciate greater flexibility in this regard.
Staff members found the free-text comments more helpful and revealing than quantitative responses, and reported that sometimes these comments provided the context and detail required for staff to learn from, and act on, patient views.
Although staff understood the reasons for retaining the anonymity of real-time feedback responders, they suggested that patients could be invited to leave their name as an option, allowing for individual issues to be followed up effectively.
Several practice managers mentioned that they were making, or intended to make, changes based on real-time feedback. Some wanted to amalgamate real-time feedback results with data from other sources before formulating a plan. Some practices planned to notify their patients about changes made on the basis of real-time feedback, whereas others involved patients in formulating these changes. Similarly, the degree of involvement of the wider practice team in action planning varied among practices. Many were keen to involve all staff in discussions (based on regular progress reports) about acting on feedback. Some individuals, however, felt remote from the decision making:
‘I haven’t been involved. I don’t know what the plan is from here.’
(016021, receptionist, interview)
These disparities relate to the differing communication cultures within practices, mentioned earlier on, which underpinned the success or otherwise of the real-time feedback implementation.
DISCUSSION
Summary
Real-time feedback represents a new approach to collecting patient feedback for primary care staff and patients. This study found that real-time feedback was received more positively by practices in which information about, and enthusiasm for, new initiatives were shared throughout the practice. This required good communication between staff groups and individuals, fostering a sense of involvement in all aspects of the implementation. In practices where messages about the rationale for real-time feedback, and the content of ongoing progress reports (from the research team) were communicated effectively, ‘buy-in’ from all staff was achieved.
Many practices viewed real-time feedback as part of ongoing communication with their patients and, in such settings, the immediacy of it helped offset ‘feedback fatigue’. Conversely, in practices where information was not communicated effectively among all staff groups, individuals, and patients, there was a feeling of remoteness from the feedback process. Greater practice involvement with the design of the survey, and actual topics covered, may achieve a greater sense of ownership, trust, and engagement among staff.
Timing of requests for patient feedback (pre/post-consultation and within the context of other practice activities) is also an important consideration and can greatly affect staff perceptions of real-time feedback and their ability and willingness to prioritise it.
Some practice staff were concerned about low real-time feedback response rates and highlighted the likelihood that some groups of patients may not be comfortable leaving feedback using the real-time devices.
Strengths and limitations
All the practices approached were located within one of two broad geographical areas, and had received scores in the lowest 50% for the GP Patient Survey communication items in the year preceding the study. The final sample included a range of settings, list sizes, and deprivation deciles. Medical, nursing, administrative, and reception staff contributed to discussions in focus groups or gave their views in one-to-one interviews.
Both qualitative researchers were from a health services research background. The project manager, with oversight of both qualitative and quantitative aspects of the real-time feedback research, contributed to regular discussions about the analysis.
The research team’s understanding of the obstacles and drivers associated with embedding real-time feedback in general practices has been enhanced by organising qualitative data according to NPT constructs. Although it is important to note that all four NPT constructs operated and were experienced concurrently, the adoption of this underpinning framework 12 has enabled a coherent view of the processes involved in implementing real-time feedback.
General practice staff acknowledged that their attitudes towards real-time feedback were influenced by the restricted availability of the kiosks (for one 12-week period only), so attitudes towards a permanent implementation may differ from the findings presented here.
Comparison with existing literature
Practice culture and communication style greatly influence the reaction of staff to new initiatives, such as real-time feedback. These study findings highlight the importance of ongoing, effective communication throughout the organisation; this enabled real-time feedback to be accepted and incorporated into practice routines and processes. Previous studies on nurses6,17,18 and guidance from the Department of Health19 have identified the importance of good communication within healthcare teams when implementing systems for collecting patient views. Other studies have found that working towards shared goals can break down barriers between professional groups and enhance communication within practice teams.20,21 The study presented here suggests, however, that pre-existing effective communication has helped to embed this new approach to collecting patient feedback within general practices.
This study concurs with existing literature about the feasibility of implementing real-time feedback systems in healthcare settings,4–7 and supports previous findings that immediate feedback may contribute to responsive action planned and taken by practices to address their patients’ concerns.15 It also confirms some of the patient groups for whom this means of feedback may not be suitable; namely, those with literacy problems, older people, and some minority ethnic groups.4–6,22,23
Implications for research and practice
More prescriptive instructions for practices ensure that all members of staff understand and are involved in developing their role with regard to real-time feedback, and are allowed time to discuss results and contribute to action planning. These are important components of successfully embedding the use of real-time feedback in general practices. This guidance may steer practices with a less inclusive culture towards greater involvement of all their staff. This study also highlights the importance of timing. Practices should:
pay attention to practice-based contingencies to take account of busy times and how patients’ time can best be utilised; and
carefully plan real-time feedback to fit with other feedback initiatives and avoid ‘over-surveying’ patients.
Further exploration of patient perspectives about real-time feedback and the development of materials for non-English speakers are important areas for future research.
Patients’ views are important. Recent research has highlighted challenges associated with developing and implementing effective techniques for capturing patients’ experiences of care; this study identifies real-time feedback as an approach that, if carefully implemented in practice, may offer the potential for addressing some of these challenges.
