Abstract
Background Emergency admissions to hospital at night and weekends are distressing for patients and disruptive for hospitals. Many of these admissions result from referrals from GP out-of-hours (OOH) providers.
Aim To compare rates of referral to hospital for doctors working OOH before and after the new general medical services contract was introduced in Bristol in 2005; to explore the attitudes of GPs to referral to hospital OOH; and to develop an understanding of the factors that influence GPs when they refer patients to hospital.
Design of study Cross-sectional comparison of admission rates; postal survey.
Setting Three OOH providers in south-west England.
Method Referral rates were compared for 234 GPs working OOH, and questionnaires explored their attitudes to risk.
Results There was no change in referral rates after the change in contract or in the greater than fourfold variation between those with the lowest and highest referral rates found previously. Female GPs made fewer home visits and had a higher referral rate for patients seen at home. One-hundred and fifty GPs responded to the survey. Logistic regression of three combined survey risk items, sex, and place of visit showed that GPs with low ‘tolerance of risk’ scores were more likely to be high referrers to hospital (P<0.001).
Conclusion GPs' threshold of risk is important for explaining variations in referral to hospital.
INTRODUCTION
Admissions to hospital are an increasing source of pressure on the NHS,1 and unplanned admissions represented 37% of hospital admissions in the UK in 2005–2006.2 Emergency admissions at night and weekends are distressing for patients and disruptive for hospitals, and many of these admissions result from referrals from GP out-of-hours (OOH) providers. Research has shown that there is a greater than fourfold variation between the top and bottom quartiles in OOH admission rates between GPs working for the same OOH service and caring for the same patient population,3 suggesting that doctors' decision making varies and plays an important part in determining admission rates. Qualitative research with the same group of OOH GPs suggests this may be due to lack of confidence, aversion to risk, feelings of isolation, and poor knowledge of alternatives to admission. The results suggest that interventions designed to address these problems may have a considerable impact on emergency admission rates.4
In 2005, the new GP contract allowed GPs to opt out of OOH care in the evenings, nights, and at weekends, and also made local primary care trusts (rather than GPs) responsible for the provision of OOH services. GPs are also able to choose their shifts and, consequently, where they work (primary care centre or making home visits).
Following the introduction of the new general medical services (GMS) contract in Bristol in January 2005, the aims of the current study were to compare the referral rates before and after the contract change; to investigate the attitudes of GPs to referral to hospital OOH; and to develop an understanding of the factors that influence GPs when they refer patients to hospital. It also aimed to develop an attitude-assessment tool to identify doctors with low and high referral rates to hospital which could be used in GP training sessions.
METHOD
Using statements derived from major themes from the qualitative study,4 a questionnaire was designed, piloted with 10 GPs in a neighbouring primary care trust, and modified based on their results. The final questionnaire comprised 41 Likert statements, 36 of which covered seven attitude dimensions of confidence and experience, tolerance of risk and uncertainty, complaints by patients or relatives, attitudes to hospital admission, patient-related factors, motivation, and organisational factors; an eighth dimension on job satisfaction had five additional questions based on previous research (Appendix 1).5 Data were also collected about age, years since qualification, type and size of practice, and what type of referrer GPs thought they were. Open-ended questions asked for GPs' opinions about alternatives to hospital which they felt might change their referral rates to hospital OOH and during their daytime work (Appendix 2).
A list of GP names was obtained from the OOH database (Adastra Software Ltd) for the three OOH providers around Bristol (one of which was the basis of the audit in 2004). Referral rates to hospital (OOH) were available for all GPs; however, only those who saw 20 or more face-to-face consultations in a 20-month time frame (1 January 2005 to 31 August 2006) were included in the survey. Questionnaires were sent to all GPs who had a current address on the OOH services databases. Reminder letters and questionnaires were sent after 3 weeks to the non-responders, and a personal telephone call made 2 weeks later by one of the research GPs for the study. Those who returned the questionnaire were paid £20.
Questionnaire data were entered onto an Excel® spreadsheet and combined with the OOH referral rates. Referral rates were expressed as a face-to-face rate and a total rate. The total rate included those contacts made by phone where they received ‘doctor advice’, whereas the face-to-face rate only included home visits and patient attendance at the primary care centre.
How this fits in
Emergency admissions to hospital are an increasing source of pressure on the NHS and many result from referrals from GP out-of-hours (OOH) providers. There is a greater than fourfold variation in OOH admission rates between GPs working for the same OOH services; research suggests that decision making varies and plays an important part in determining admission rates. GPs with a lower tolerance of risk are more likely to be high referrers to hospital. Providing a GP with special interest in primary care OOH service who could access alternatives to admission might enable GPs to alter their referral practices.
Statistical methods
The analysis was carried out in two parts. First, an analysis of all the GPs on the database modelled whether the associations found were similar to those seen before the contract changed.3 For this the GPs were banded into quartiles, based on their admission rates, and logistic regression was used to determine the odds of being admitted to hospital if seen by a GP in the top quartile, compared with that of being seen by a GP in the bottom quartile. Logistic regression was based on patient contacts, and STATA (version 9) accounted for multiple contacts per GP using clustering by GP. Linear regression explored any associations found, including place of visit and sex differences, with their referral rates to hospital. Comparison of referral rates for individual GPs with data available before and after the contract changed was also made using Spearman's rank correlations and Wilcoxon signed rank tests.
The second analysis involved only the GPs who had returned questionnaires. Comparisons for those who did and did not return questionnaires were carried out using χ2 test for sex and Mann–Whitney U tests for the number of contacts and referral rates. The correlation between the type of referrer that the GP perceived themselves to be, and their actual referral pattern was examined using Kendall's tau-b.
Factor analysis was used to investigate the 41 attitude statements, to determine whether there were any distinct subscales similar to those used in the design of the questionnaire on attitudes to referral to hospital from OOH providers. Different rotations were considered, including varimax and direct oblimin, to explore the clinical relevance of the subscales. Cronbach's alpha was used to identify potentially redundant items. The statements were also investigated individually to see if any of them were independently associated with referral rates.
Attitude scores were calculated for the eight questionnaire attitude dimensions, by summation of the items from each dimension. Items that were negatively correlated with the others were reversed and Spearman's rank correlation coefficients calculated to examine the association between referral rates and the eight dimensions. Similar correlations were calculated for individual items and referral rates using a square root transformation of referral rates to produce a normal distribution. Linear regression modelling was then carried out using the statistically significant items combined as a risk score with sex and place of visit.
Where the individual items were used, a statistical significance level of 0.01 was used (rather than 0.05) to reduce the chance of a type I error. A formal Bonferroni correction was not used as this was too stringent given the hypothesis-generating nature of the study. For regression modelling, P<0.05 significance was retained.
RESULTS
Data available from the Adastra OOH database for three OOH providers covering the 20-month period since January 2005 included 77 748 patient contacts with 496 OOH workers. After excluding those with no face-to-face contacts recorded (n = 37), those who had fewer than 20 face-to-face contacts (n = 212), those who were not GPs (n = 6), and a further seven duplicate names, 234 names remained. These GPs had 73 453 total patient contacts (n = 51 688 face-to-face), with 6353 admissions to hospital (n = 4913 from face-to-face contacts).
Admission rates
Admission rates were available for 234 GPs who had from 20 to 2580 face-to-face patient consultations. Face-to-face admission rates varied from 0% for a GP who had 26 face-to-face consultations, to 38.5% for a GP who had 91 face-to-face consultations. Referral rates were compared between the highest and lowest quartiles both before and after the change in contract. Table 1 shows that since the change in contract the odds ratio for being admitted to hospital if seen by a GP in the highest quartile was 4.49 (95% confidence interval [CI] = 3.98 to 5.06) compared with being seen by a GP in the lowest quartile, which was similar to odds ratios for the 3 years prior to the change in contract. The referral rates were also very similar, with the overall rates being 9.54% before the change in contract and 9.50% after the contract changed.
Comparison of being admitted to hospital if seen by a GP in each of the rates of admission quartiles from face-to-face contacts before and after introduction of the new general medical services contract.a
Referral rates were also compared for 80 individual GPs with data available for both periods of time. There was a significant correlation in their referral rates (Spearman's r = 0.57, P<0.001) before and after the contract changed. Median referral rates for this group of GPs were not significantly different (Wilcoxon P = 0.17) before (9.2%) and after (9.8%) the contact changed. There were no significant differences in the referral rates of those who continued with OOH work and those who did not (Mann–Whitney P = 0.887). Although fewer female GPs continued with the new contract (44% compared to 51% before), the proportion of males to females was not significantly different.
There was a difference in the number of patients seen at home or the primary care centre for male and female doctors, in that females made fewer home visits (16% of contacts) compared with males (21%) and seemed to choose more shifts based at the primary care centre. The distribution of referral rates for male and female doctors by the site of the consultation (primary care centre or home) was also different, as shown in Table 2, being highest for females when visiting patients at home. Linear regression modelling of referral rates controlling for place of visit showed that these sex differences were statistically significant (P<0.001), indicating that female GPs referred more patients to hospital than male GPs.
Referral rate (%) to hospital out of hours by GP sex and place of consultation.
Postal survey
Of the 234 names, 17 were excluded as there was no current address, and another 23 were only registered with one OOH provider that did not distribute the questionnaires, leaving an eligible total of 194 GPs. Six of these returned their questionnaires indicating that they did not wish to take part, and 38 did not return a questionnaire, resulting in 150 completed questionnaires (77.3% return rate). There were no differences in sex or referral rates between those who did and did not return their questionnaires.
There was a significant correlation (P = 0.002, Kendall's tau-b = 0.21) between what type of referrer (high, medium, low) GPs thought they were and their referral rates, but interestingly only 5% (n = 7) of them felt that they were high referrers. The correlation was low, indicating that there was a general lack of awareness about their own referral rate compared to those of others.
Factor analysis did not produce any clinically relevant subscales even after trying different rotations. The confirmatory factor analysis of the eight attitude dimensions suggested that there might be further subscales present within some of the dimensions, but due to the low number of items within each dimension it was not possible to identify stable subscales within them. Cronbach's alpha did not indicate that the omission of any items would lead to a substantial improvement.
Spearman's rank correlation showed no statistically significant associations between any of the eight dimensions of the attitude survey and referral patterns, as shown in Table 3. A sample of 150 participants has 80% power to detect correlations of 0.23 or greater as statistically significant at the 0.05 level, and none of these correlations was greater than 0.13.
Correlations between questionnaire attitude dimensions and referral rates (n = 150).
Analysis of the 41 individual items produced three items from the tolerance of risk and uncertainty dimension with statistically significant correlations with referral rates as displayed in Table 4. These indicated that GPs who believe that they are cautious have higher referral rates; those who believe they are carrying out risk assessment and those who believe they are good at living with uncertainty and risk both have lower referral rates.
Item-by-item correlation coefficients and P-valuesa for the 41 attitude items with referral rates (both total admitted rates and rates for those admitted following face-to-face consultations).
A risk score was computed by summing these responses (items 4, 7, and 8) related to tolerance of risk and uncertainty. Linear regression modelling transformed the face-to-face referral rate with sex, controlling for place of visit, showed that female doctors were significantly more likely to refer patients to hospital than male doctors (P = 0.039). However, when the risk score was included in the model, the sex difference was no longer significant and the risk score had a stronger association with referral rate (P<0.001), indicating that it is GPs with a lower tolerance of risk that are more likely to admit patients to hospital OOH.
Free-text comments on the questionnaires suggested that GPs might change their referral behaviour if there were better access to intermediate care, more information about patients, and some follow-up or feedback about patients.
DISCUSSION
Summary of main findings
This study has confirmed the authors' previous finding3 with the current group of 234 GPs working for three OOH organisations, that there remains a greater than fourfold variation in referral rates between the highest and lowest quartiles of GPs since January 2005. Referral rates have also not changed since the new GMS contract started. This is remarkable, since the study was comparing three post-contract OOH organisations with one pre-contract one and a different group of GPs, who had all opted to do OOH work. Previous findings from the authors' qualitative study,4 which suggested that GP perception of risk had an influence on decisions to refer, were also confirmed. The survey evidence showed that those who are comfortable living with risk are likely to be low referrers to hospital, and this was also evident in the qualitative study where high-referring GPs were typically cautious, believed it was better to refer if in doubt, and expressed anxiety about the consequences of a decision not to admit, for both the patient and themselves.4 The authors had believed that the changes in the new contract would attract different types of GPs to OOH for different reasons and motives, and that those who were not comfortable with it would opt out. However, this was not reflected by GPs' referral decisions.
Strengths and limitations of the study
The strength of the study is the high response rate in the survey from those who received it. Thus the results can be considered as representative of the views of GPs working in OOH services. Limitations include the use of only three GP OOH services and the relatively large numbers of items that were used to try to assess whether attitudes were associated with referral behaviour. While a large majority of these attitudinal items were derived directly from the qualitative study, their validity as a measure had not been evaluated in previous research. Also, the small number of items that were statistically significant did not enable the development of a comprehensive assessment training tool as initially intended. However, the analysis did identify three risk items that might be used as an ‘attitude to risk’ tool to predict referral rates.
Comparison with existing literature
This study showed that female GPs refer more patients to hospital OOH, and they seemed to choose shifts based in the primary care centre rather than doing home visits. When this was allowed for in the analysis, GPs with low scores for tolerance of risk were more likely to be high referrers to hospital. This supports research evidence from daytime referral practices,6 which suggests GPs' threshold of risk is important for explaining variations in referral. However, males and females work in different ways, with females being more patient centred,7 and perhaps more risk averse. On the other hand, male doctors have a higher output than females, but this is offset by males experiencing more litigation and disciplinary action.8
Most GPs seemed to know what type of referrer they were, but a few felt that they were lower referrers than was actually seen in their referral rates. They indicated, as was evident in the previous study,4 that they might alter their referral practices if reliable and accessible alternatives to hospital were readily available to keep patients at home, while additional training, feedback, and opportunities for a second opinion without referring might also affect some GPs' referral decisions.
Understanding how different doctors make decisions,9 and the difficulties they have estimating and managing risks,10 is important in planning changes in their performance. Different doctors have different ways of managing uncertainty and may be unaware of how they compare with others; poor performers often have misplaced confidence in their performance.11 Giving doctors educational feedback on their performance compared to the optimal can change the clinical performance of some doctors,11 and may have an impact on referral rates OOH.
Implications for future research or clinical practice
Some GPs have suggested that a separate defined OOH specialty with recognised qualifications and bespoke or mandatory training in primary care OOH services might provide a better service.12 An ‘attitude to risk’ assessment tool could be used in such training.
Further research could explore in greater depth why female GPs may be more risk averse. Another important area to investigate would be how to facilitate access to intermediate care or other alternatives to emergency admission, and feedback to GPs about patient outcomes, to provide a greater insight into what OOH practitioners would value and whether patients' quality of life could be improved.
Acknowledgments
We thank all the GPs who took part, Rob Simpson for managing the Adastra data, and Hannah Spiers for questionnaire data management.
Appendix 1. Attitudes to referral to hospital from out-of-hours questionnaire.
The following statements, some of which have been made by GPs, represent a range of views about the influences on decisions to refer to hospital or not from out-of-hours (OOH). Please indicate how far you agree with these statements by selecting the appropriate level of agreement.
Appendix 2. Background factors questionnaire.
Please complete the following or circle the appropriate alternative.
aAbbreviations ‘Brisdoc’, ‘Frendoc’, and ‘Nordoc’ are used by three local out-of-hours services covering three areas of the city of Bristol.
Notes
Funding body
Avon Primary Care Research Collaborative small grant scheme (05-IPE4)
Ethical approval
Southmead Research Ethics Committee (ref 05/Q2002/62) August 2005
Competing interests
The authors have stated that there are none
Discuss this article
Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss
- Received February 6, 2008.
- Revision received April 4, 2008.
- Accepted June 30, 2008.
- © British Journal of General Practice, 2009.
In this issue
Jump to section
More in this TOC Section
Related Articles
Cited By...
