Intended for healthcare professionals

General Practice

Postal survey of patients' satisfaction with a general practice out of hours cooperative

BMJ 1997; 314 doi: https://doi.org/10.1136/bmj.314.7094.1594 (Published 31 May 1997) Cite this as: BMJ 1997;314:1594
  1. Chris Salisbury, senior lecturer in general practicea
  1. a Department of Primary Health Care and General Practice Imperial College School of Medicine at St Mary's London W2 1PG
  • Accepted 13 May 1997

Abstract

Objective:To assess patients' satisfaction with out of hours care by a general practice cooperative compared with that by a deputising service.

Design:Postal questionnaire survey.

Setting:A general practice cooperative in London and a deputising service operating in an overlapping area.

Subjects:Weighted samples of patients receiving telephone advice, a home visit, or attending a primary care centre after contacting either service in an eight week period.

Main outcome measures:Patients' overall satisfaction and scores for specific aspects of satisfaction. Satisfaction with telephone advice or attendance at centre compared with home visit. Relation between satisfaction and patient's age, sex, ethnic group, car ownership, preference for consulting own doctor, and expectation of a visit.

Results:The overall response rate was 67% (1555/2312). There was little difference in overall satisfaction between patients contacting the cooperative or the deputising service, but patients contacting the latter were less satisfied with the explanation and advice received and the wait for a visit. There were significant differences between patients in different age and ethnic groups, with white patients and those aged over 60 years being more satisfied. Lower scores for overall satisfaction were reported by patients who received telephone advice, those who would have preferred to see their own doctor or who originally wanted a home visit, and those who waited longer for their consultation. Overall levels of patients' satisfaction seemed to be lower than previously reported.

Conclusions:There were larger differences in satisfaction between different groups of patients than between different models of organisation for out of hours care. A shift to a service based predominantly on telephone advice may lead to increased patient dissatisfaction.

Key messages

  • Recent growth in general practice cooperatives for out of hours care has changed the way that care is delivered, and this study assessed patients' satisfaction with a general practice cooperative compared with a deputising service

  • There was little difference in overall satisfaction between patients contacting the cooperative or the deputising service, but a substantial minority of patients expressed dissatisfaction with some aspects of both out of hours services

  • Patients given telephone advice, especially when they had wanted a visit, and those from non-white ethnic groups were particularly likely to be dissatisfied

  • Lower scores for overall satisfaction were also found in younger patients, those who would have preferred to see their own doctor, and those who waited longer for their consultation

  • If out of hours services are based predominantly on telephone advice patients' satisfaction may decline

Introduction

The recent growth in general practice cooperatives for out of hours care has been accompanied by important changes in the way that care is delivered. Many cooperatives have established primary care centres to which patients can be invited and are offering telephone advice from a doctor as an alternative to a home visit.

The pressure for change in the system has come almost entirely from the medical profession, and it is important to consider the patients' perspective. A radical restructuring of out of hours services in Denmark, which followed problems similar to those occurring in the United Kingdom, showed a decline in patients' satisfaction after the reorganisation, although the new scheme was considered a success by doctors and health authorities.1 It has been claimed that a high proportion of out of hours callers can be advised over the telephone,2 but there is no evidence that patients are satisfied with this arrangement. A study of out of hours primary care centres showed that those patients who attended were generally satisfied, but only 22% of callers were willing to attend, mainly because of unavailability of transport.3

Several cooperatives have claimed high levels of patient satisfaction, but the results have not been published.4 Assessing patients' satisfaction is complex,5 and the use of valid and reliable measures is important.6 Apparent high levels of satisfaction may reflect the insensitivity of the measures used.7

This study of patients' satisfaction forms part of a project to evaluate a general practice out of hours cooperative in the Kensington, Chelsea, and Westminster area of London by comparison with a commercial deputising service operating in this area and the neighbouring area of Brent and Harrow. Details of the setting have been described previously.8 The survey was designed to make comparisons between the satisfaction of patients calling the cooperative and the deputising service, and between patients receiving telephone advice, a home visit, or attending the primary care centre.

Method

Development of the questionnaire

The instrument for this study was based on minor modifications to a questionnaire developed by McKinley and colleagues.9 This was devised with a principal components analysis to assess several aspects of satisfaction with consultations outside normal surgery hours. The questionnaire consists of questions grouped in themes, each requiring a response on a five point scale. Although some changes to McKinley's questionnaire were necessary for this project, alterations were kept to a minimum.

The original questionnaire is not suitable for patients attending a primary care centre. I therefore produced three separate versions of the questionnaire that were identical except for one section, which asked questions applicable only to patients receiving telephone advice, a home visit, or attending the centre. In this way it was possible to avoid complex skip sections which lengthen the questionnaire and reduce the response rate.7 A set of questions about whether patients preferred to see their own doctor was simplified to one question as this was not an important issue for this study. Because I had modified the original questionnaire and was using it in a different setting, I had to carry out my own principal components analysis and reliability assessment on the modified questionnaire.

Power of the study

Weighted samples were randomly selected from all those patients contacting a doctor at the cooperative or deputising service over an eight week period starting 1 September 1995. Different weightings were used to select similar numbers of patients who had received different types of contact (telephone advice, home visit, centre attendance). The sample size was adequate to detect a difference in satisfaction score of a quarter of a standard deviation between groups each containing at least 250 patients at a significance level of 0.05 and 80% power.

Procedures

Patients were sent a postal questionnaire with a covering letter and reply paid envelope within seven days of their contact with the cooperative or deputising service. Reminders were sent to non-responders after a further 10 and 20 days. Patients were excluded if it was evident that they had died, been admitted to hospital under the Mental Health Act, were demented and living in a nursing home, were visiting the area for less than a week, or had already been sent a questionnaire in an earlier contact. Questionnaires were identified by code number, which enabled responses to be related to information obtained from other aspects of the study.

Statistical analysis

Data were entered with Epi-Info and analysed with SPSS for Windows and SAS statistical packages. Questions were scored from 1 to 5, where 5 represented strong agreement with a statement of satisfaction, 3 was neutral, and 1 represented strong disagreement. For negatively worded items, the scores were reversed so that a high score always indicated satisfaction. The weightings given to the subgroups in the sample were used throughout the analysis (see table 1). I carried out a principal components analysis using a Varimax rotation to derive scales for different aspects of patients' satisfaction, which were compared with those from the original questionnaire. The score for each patient on each scale of satisfaction represented their average response to the questions that made up that scale.

Table 1

Details of patients' contacts with general practice out of hours service

View this table:

I used weighted multiple regression analysis to compare satisfaction with the cooperative or deputising service after allowing for confounding variables of age, sex, and ethnic group. These variables were entered simultaneously. As the effect of age on satisfaction seemed to be non-linear, separate variables were created for different age groups. In a further analysis several other variables, such as the setting for the consultation and whether patients would have preferred to see their own doctor, were incorporated in the analysis to assess the effect of these factors on patients' satisfaction.

The internal reliability of each scale was assessed with Cronbach's α coefficient.

Results

Background characteristics

The overall response rate was 67% (1555/2312). Table 1) gives details of response rates and weightings used in the analysis.

There were no significant differences between respondents and non-respondents in terms of age and sex. Among respondents the patients contacting the cooperative were older (median age 28 (interquartile range 4-50) years) than those who contacted the deputising service (21 (4-47) years). In terms of sex ratio, 62% (646/1036) of cooperative patients were female compared with 57% (297/517) of deputising service patients (weighted odds ratio 1.08 (95% confidence interval 0.86 to 1.36), adjusted for age and ethnicity). A significantly higher proportion of patients consulting the deputising service (223/507, 44%) came from non-white ethnic groups than did those consulting the cooperative (256/1007, 25%) (weighted odds ratio 2.29 (1.82 to 2.89), adjusted for age and sex).

Patients' satisfaction

Principal components analysis identified five aspects or “scales” of satisfaction, which were labelled “explanation and advice,” “doctor's manner,” “contacting the service,” “receptionist,” and “overall satisfaction.” The three versions of the questionnaire for patients receiving home visits, telephone advice, or attending the centre each had one section of questions that were similar but specific to the type of patient contact. This section was analysed separately, and two further scales were identified for each version of the questionnaire (see table 2). Details of the individual questions which make up each scale are available from me. The scales proved similar to those identified by McKinley et al,9 but were not identical, mainly due to the separate section of questions applicable to different types of patient contact. All scales proved reliable, with α coefficients between 0.68 and 0.92.

Table 2

Comparison of patients' satisfaction with out of hours care by general practice cooperative and deputising service

View this table:

Patients contacting the cooperative were less likely to want a visit from the doctor (506/1033, 49%) than those contacting the deputising service (395/511, 77%) (weighted odds ratio 0.29 (0.23 to 0.38) adjusted for age, sex, and ethnicity). More than a third (249/686, 36%) of those patients given telephone advice had originally wanted a visit, and these patients had a mean score on the overall satisfaction scale of 2.63, indicating dissatisfaction. More than a third of all respondents (547/1531, 36%) would have preferred to see or speak to their own general practitioner rather than contact the out of hours service; the remainder had no preference.

Table 2) shows the scores for each scale of patients' satisfaction for the cooperative and the deputising service, and shows the results of multiple regression analysis to compare the services after adjustment for patients' age, sex, and ethnic group. There was little difference in overall satisfaction between patients contacting the cooperative or deputising service, but patients of the latter were less satisfied with the doctor's explanation and advice and with the wait for a visit.

Table 3) shows the effect of other variables on overall satisfaction. Patients from non-white ethnic groups were significantly less satisfied with the out of hours service, whereas satisfaction increased with age. Only the results for overall satisfaction are shown, but these findings also applied to the other subscales. Overall satisfaction declined the longer that patients waited for their consultation. Patients who originally wanted a home visit and those who would have preferred to see their own doctor were also less satisfied overall. Patients who received telephone advice were less satisfied than those who received a visit or attended the centre. Patients attending the primary care centre were as satisfied as those receiving visits. The small number of patients attending the centre may be because only 41% (417/1025) of respondents had easy access to a car at the time they contacted a doctor.

Table 3

Weighted least squares multiple regression analysis to show effect of variable on patients' overall satisfaction with general practice out of hours care

View this table:

Although there were few differences between the cooperative and the deputising service in terms of patients' satisfaction, the answers to individual questions revealed high levels of dissatisfaction with some aspects of both services. Thirty eight per cent (571/1499) of patients felt that “the arrangements for contacting the doctor could be improved,” and 33% (355/1074) “would have liked the doctor to tell them more about their treatment.” Of those who received visits, 36% (265/739) were “worried because it took a long time for the doctor to arrive.” Of those given telephone advice, 25% (163/654) were “a little unhappy with the telephone advice they received” and 49% (318/655) “would have preferred to have had a visit.” Forty three per cent (638/1474) felt that the out of hours service could be improved.

Discussion

Patients' satisfaction is not necessarily the main criterion by which primary care services should be judged, but the attitudes of the consumers of health care are an important factor which must be considered in evaluating services. The results presented here suggest that, as in Denmark,1 changes that are seen as beneficial by doctors are not necessarily welcomed by patients.

The recent contractual changes for general practitioners, allowing them to determine whether and where an out of hours consultation should occur, has led to an increasing emphasis on telephone advice and surgery based care. This study shows that telephone advice is a common cause of patient dissatisfaction. Patients who attended the primary care centre were generally satisfied, but few patients accepted this option. Many people apparently did not have access to a car. This finding, which supports earlier research,3 suggests that it may be overly optimistic to assume that most home visits can be replaced by surgery consultations unless the problem of responsibility for transport is addressed.

The aim of this research was to determine whether patients consulting a general practice cooperative were more or less satisfied than those consulting a deputising service. In fact, there were larger differences in satisfaction between different groups of patients than there were between these different models of out of hours service. The dissatisfaction expressed by patients from ethnic minority groups corresponds with the results of an earlier study, which found that Asian patients placed great importance on house visits and disliked management by telephone.10 The finding that satisfaction increased with age is a also a consistent finding of patient surveys.11 12 13 It is clearly important to consider the ethnic and age structure of the local population when interpreting the results of future surveys of patients' satisfaction with out of hours services.

The level of dissatisfaction identified in this study seems higher than in previous reports, with mean scores for most scales being only slightly above the midpoint between satisfaction and dissatisfaction. There are several possible reasons for this. Firstly, the most widely quoted studies of satisfaction with deputising services were carried out in 1986 and 1987,12 13 since when patients' expectations may have changed. Secondly, most unpublished surveys carried out locally by general practitioner cooperatives have used simple “yes or no” questionnaires to ask patients whether they are satisfied. Such questionnaires, particularly if sent back to the service concerned, are likely to produce artificially high levels of satisfaction, as patients may feel diffident about criticising a service so directly.11 Thirdly, London contains a high proportion of patients from ethnic minority groups, and this has been shown to have a large effect on satisfaction.

What conclusions should be drawn from the apparent conflict between the wish of general practitioners to change the out of hours service and the concerns of many patients about these developments? Consumers' representatives should be involved in planning new arrangements in out of hours care, and the reasons for changes should be discussed. Patients' satisfaction is strongly influenced by prior experience and what services are currently available,14 so some dissatisfaction might be anticipated during a time of change. Changes in the delivery of care might be introduced gradually in order to reduce this effect.

This research has shown the level of patients' satisfaction with a general practice cooperative and a deputising service. The results may not be generalisable to other areas, particularly outside London, but they offer a basis for comparison for other out of hours services which wish to assess patients' satisfaction in a rigorous manner.

Acknowledgments

I thank Dr R McKinley for allowing me to adapt his patient satisfaction questionnaire; Ms S Mandalia for statistical advice; Dr S Hargreaves, Mr M Henchey, Dr A Dicker, and Dr N Kaiper Holmes for their help with this project; the staff and patients at both services; and my research assistant, Anna-Marie Hill.

Funding: This project was funded jointly by Kensington, Chelsea, and Westminster Medical Audit Advisory Group and Healthcall plc.

Conflict of interest: Both organisations funding this study have an interest in the results. However, neither had any part in the design, conduct, analysis, or presentation of this work, which are entirely the responsibility of CS.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.