Article Text
PDF
Abstract
Background Patient satisfaction and experiences are important parts of healthcare quality, but patient expectations are seldom included in quality assessments. The objective of this study was to estimate the effects of different predictors of overall patient satisfaction with hospitals, including patient-reported experiences, fulfilment of patient expectations and socio-demographic variables.
Methods Data were collected using a national patient-experience survey of 63 hospitals in the five health regions in Norway during the autumn of 2006. Postal questionnaires were mailed to 24 141 patients after their discharge from hospital. Non-respondents were sent a reminder after 4 weeks. Multivariate linear regression analysis including multilevel regression was used to assess the predictors of overall patient satisfaction with hospitals.
Results Thirteen variables were significantly associated with overall patient satisfaction: two variables about fulfilment of expectations, eight about patient-reported experiences and three socio-demographic variables. The regression model explained 59% of the variation in overall patient satisfaction. The most important predictor of patient satisfaction with hospitals was patient-reported experiences with the nursing services (β=0.27, p<0.001), followed by fulfilment of patient expectations (β=0.21, p<0.001), experiences with doctor services (β=0.12, p<0.001) and perceived incorrect treatment (β=−0.12, p<0.001). Multilevel regression analysis confirmed most of the findings, but revealed that age was not a significant predictor of overall patient satisfaction.
Conclusions The study showed that both fulfilment of expectations and patient-reported experiences are distinct from but related to overall patient satisfaction. The most important predictors for overall patient satisfaction with hospitals are patient-reported experiences and fulfilment of expectations.
- Patient satisfaction
- questionnaire
- quality indicator
- hospital
- patient-centred care
- performance measures
- quality measurement
- statistics
Statistics from Altmetric.com
- Patient satisfaction
- questionnaire
- quality indicator
- hospital
- patient-centred care
- performance measures
- quality measurement
- statistics
Background
There is no consensus in the literature as to how to define and measure the patient perspective on healthcare. Four different approaches have been described in a systematic review of the patient-satisfaction literature: approaches based on expectations; approaches based on health-service attributes; economic approaches; and holistic approaches.1 These approaches differ in various ways. For example, expectation-based approaches focus on the association between expectations, perceived experiences and patient satisfaction, while the health-service attribute approach normally excludes satisfaction and expectations, instead focusing on patient-reported experiences on different health-service factors. Holistic approaches try to include all important predictors of patient satisfaction, thus providing a comprehensive framework for exploring interactions between variables that affect consumer evaluations.1
One important application of patient evaluation is in the use of quality indicators.2 3 In Donabedian's classical quality-measurement approach, patient satisfaction is included as an outcome measure together with changes in health, knowledge and behaviour.2 The model also includes healthcare structures (conditions) and processes of care (activities), and requires a causal link between structure, process and outcome.2 Following this model, patient satisfaction can be defined as a patient-reported outcome measure, while the structures and processes of care can be measured by patient-reported experiences. The causal link between structure, process and outcome might be expected theoretically, provided that patients are asked relevant and important questions about their healthcare experiences. This should be secured by a rigorous process of questionnaire development and validation. However, patient satisfaction is based on a subjective evaluation, and the holistic approach described above predicts that both actual experiences, patients' evaluations of these experiences, and other individual factors might affect patient satisfaction. In theory, this problem also relates to the concept of patient-reported experiences with the structures and processes of care, which necessitates some element of subjective evaluation. Consequently, the effects related to factors at the individual level should be estimated and accounted for when using patient evaluation in quality measurement.
Various individual-level factors might affect patient satisfaction. In addition to known socio-demographic predictors such as age and health,1 patient expectations are often described as a major determinant of patient satisfaction.1 4 There is no consensus about how to define and measure patient expectations.5 One theoretical approach divides expectations into three different types: predictive expectations—likely performance on attributes; normative expectations—desired performance on attributes; and comparative expectations—expectations from similar products/services.6 In addition to being a subjective construct, this implies that expectations should be defined and measured as a multidimensional concept. It also means that expectations should be included in models aiming to explain variations in patient satisfaction. However, a review of the literature reveals that only a small proportion of the patient-satisfaction research conducted so far has included patient expectations.1
The objective of this study was to estimate the effects of different predictors of patient satisfaction with hospitals, including patient-reported experiences, patient expectations and socio-demographic variables. The Norwegian Knowledge Centre for the Health Services conducted a national postal patient-experience survey among adult inpatients discharged from Norwegian hospitals in 2006. The data set included survey data about patient-reported expectations, experiences and satisfaction, and administrative data about patients from the hospitals. The dependent variable was an overall patient-satisfaction item, representing the outcome measure according to Donabedian's model.2 Patient-reported experiences were included via an eight-dimensional scale covering patient evaluation of the structures and processes of care, in addition to several additional single items on patient experiences. Various socio-demographic variables were also included, such as age and health, in addition to two items about fulfilment of expectations. The expectation items required patients to report whether their experiences were better, worse or the same as expected. Finally, we adjusted for the hospital level through multilevel analysis in order to avoid biased estimates for individual-level factors. We are not aware of any other studies on overall patient satisfaction that have included both fulfilment of expectations and patient-reported experiences as predictors, and used multilevel analysis to estimate and control for the hospital-level effect.
Based on theory and existing empirical studies, we developed the following three hypotheses:
Patient-reported experiences are significantly but modestly correlated with overall patient satisfaction (related but distinct constructs).
Fulfilment of expectations is significantly but modestly correlated with overall patient satisfaction (related but distinct constructs).
Patient-reported experiences and fulfilment of expectations are more important predictors for overall patient satisfaction than socio-demographic factors.
Methods
Data collection
The national survey included adult inpatients discharged from Norwegian hospitals between 1 September and 23 November 2006. We selected a random sample of 400 patients for each of the 63 hospitals, or included all eligible patients during the sampling period if the number of patients was less than 400. Maternity wards, psychiatric units and children treated at adult departments were excluded from the survey. Non-respondents were sent a postal reminder after 4 weeks. In total, 24 141 patients were included in the study; 345 patients were not eligible. All hospitals transferred data about the included patients—including age, gender, admission type, length of stay and diagnosis—to the Knowledge Centre. Test–retest reliability was assessed by sending a second questionnaire to a sample of 270 patients approximately 1 week after they had returned the first one.
Questionnaire
The questionnaire comprised 26 items about patient experiences, two items about expectations, one item about overall patient satisfaction, 14 questions about quality of life and 10 background questions. The overall patient-satisfaction question was ‘All in all, were the care and treatment you received at the hospital satisfactory?’, with a five-point response format ranging from ‘not at all’ to ‘to a very large extent’. The test–retest reliability of this single item was 0.73 (n=184). The two questions about expectations comprised an overall assessment of the hospital against expectations, and perceived impact of the hospital stay on health against expectations. Both questions had a five-point response format ranging from ‘much worse than expected’ to ‘much better than expected’. The test–retest reliabilities of these two items were 0.71 (n=187) and 0.61 (n=181), respectively. The level of agreement between the two sets of scores was assessed using the intraclass correlation coefficient (ICC).
The patient-experience questions were based on the Patient Experiences Questionnaire,7 but the response scale was changed to improve the data quality.8 All of the experience items used a five-point response format ranging from ‘not at all’ to ‘to a very large extent’. The national report used the following six scales, for which there was good evidence for data quality, reliability and validity:9 doctor services (three items), nursing services (four items), information examinations (two items), organisation (three items), hospital and equipment (two items), and contact with next of kin (two items).
Two initiatives were carried out in order to avoid extensive loss of information in the regressions. First, two additional patient-experience scales were created, incorporating items that were left out of the national report. The first scale related to items concerning both doctors and nurses; it included three items and had a satisfactory internal consistency reliability (Cronbach's α=0.81). The other scale included two items about information on future complaints (Cronbach's α=0.85). Second, items with a large amount of missing data were recoded into dichotomous variables: the most negative categories; and the remaining categories including the missing data. This related to items about pain relief, contact with next of kin, medicines and corridor stay. For these variables negative experiences are expected to be related to patient satisfaction, while positive experiences and no experiences are expected to be unrelated to patient satisfaction.
Consequently, the following patient-experience scales and items were used in the regression analysis: doctor services (scale with three items), nursing services (scale with four items), information examinations (scale with two items), organisation (scale with three items), hospital and equipment (scale with two items), health personnel in general (scale with three items), information on future complaints (scale with two items), incorrect treatment (one item), unforeseen waiting (one item), pain relief (one item), contact with next-of-kin (two items), medicines (two items) and corridor stay (one item). Scale scores were transformed linearly to a scale of 0–100, where 100 is the best possible rating, while items were scored 1–5, where 5 is the best rating.
Statistical analysis
Correlations between overall patient satisfaction, patient-reported experiences and patient expectations were tested using Pearson's r. Multivariate linear regression analysis was used to assess the effects of patient-reported experiences and expectations on overall patient satisfaction, controlling for gender, age, self-perceived health, education, admission type, number of admissions in the previous 2 years and length of hospital stay.
The effect of patient clustering within hospitals was tested using multilevel linear regression. Patient clustering might inflate t values in ordinary linear regression models, possibly resulting in a type I error. Multilevel modelling is recommended when testing the effects of group-level variables,10 since it specifically divides the total variance in patient-reported experiences into variance at the hospital (macro) versus the patient (micro) level. The ICC in an empty model with overall patient satisfaction as a dependent variable and hospitals as random intercept was 0.038. The design effect was higher than the criteria of 2,11 indicating a need for multilevel modelling. The hospitals were included as random intercepts, and all variables from the ordinary regression were included as fixed effects at the patient level. Standardised variables at level 1 were used in the regression, and consequently standardised regression coefficients were computed. SPSS V.15.0 was used for the statistical analyses.
Results
The questionnaire was answered by 10 912 patients (response rate 46%). The overall patient-satisfaction item was skewed towards positive assessment: 4.2 on a scale of 1–5, where 5 represents the best score (table 1). Of those who responded, 38.9% were satisfied with the hospital to a very large extent, 49.2% to a large extent and 9.2% to some extent. Only 2.0% reported to be satisfied to only a small extent, and 0.7% were not at all satisfied with the hospital.
Univariate results: overall patient satisfaction, patient-reported experiences and fulfilment of expectations
A large proportion of patients reported that their experiences matched their expectations: 44.3% reported that the hospital and hospital stay were as expected (table 1), while 25.2% reported that the hospital was much better than expected, and 23.3% reported the hospital to be somewhat better than expected. Only 1.5% answered that the hospital was much poorer than expected, and 5.7% reported that the hospital was somewhat poorer than expected. The health effects of the hospital stay were reported to be as expected by 55.4% of the respondents (table 1), much better than expected by 18.4% and much poorer than expected by 2.8% (table 1). Scores on the patient-reported experience scales ranged from 47.1 (information about post-discharge) to 79.9 (nursing services), on a scale from 0–100, where 100 represents the best score (table 1).
All correlations between overall patient satisfaction, patient expectations and patient-reported experiences were significant at the 0.01 level (table 2). The strongest correlation was found between two patient-reported experience scales: the correlation between doctor services and health personnel in general was 0.73. The weakest correlation was between the patient-reported experience scale relating to hospital and equipment and the item about health effects versus expectations (0.24).
Correlations between overall patient satisfaction, fulfilment of expectations and patient-reported experiences*
Ordinary multivariate regression analysis revealed that 13 variables were associated with overall patient satisfaction (table 3): two items about patient expectations, eight patient-reported experience variables and three background variables. The strongest predictors were patient-reported experiences with nursing services (β=0.267), hospital experiences versus expectations (β=0.212), doctor services (β=0.121) and perceived incorrect treatment (β=−0.118). The regression model explained 59% of the variance in overall patient satisfaction.
Multivariate linear regression models: associations between independent variables and overall patient satisfaction
Multilevel regression analysis confirmed the findings of the ordinary regression analysis (table 3), but the regression coefficients were lower after taking into account the effect of patient clustering within hospitals, and age was found to not be a significant predictor. The pseudo explained variance in the multilevel model was the same as for the ordinary regression (ie, 59%).
Discussion
Studies of patient satisfaction should include the most important sources of variation in satisfaction. Expectations are an important predictor in conceptual models of patient satisfaction, but the effect of expectations on patient satisfaction is seldom assessed empirically.1 The present study included both patient-reported experiences and fulfilment of expectations as predictors of overall patient satisfaction, while simultaneously controlling for hospital-level effects and socio-demographic variables. The main study hypotheses were supported. First, patient-reported experiences and fulfilment of expectations were the most important predictors of overall patient satisfaction. Second, both patient-reported experiences and fulfilment of expectations were moderately associated with overall patient satisfaction in bivariate analysis, giving support to the hypothesis that these are related but distinct constructs.
Following Donabedian's model,2 overall patient satisfaction was defined as a patient-reported outcome measure in this study, while structures and processes were represented by patient-reported experiences. A causal link was established between eight patient-reported experience aspects and overall satisfaction, but seven experience aspects were not related to overall satisfaction. This implies that the latter aspects might be of less relevance to quality-improvement processes in hospitals, provided that the goal is to improve overall patient satisfaction. Furthermore, the large effect of fulfilment of expectations shows that subjective factors are important sources of bias when using patient satisfaction in quality measurement. These subjective factors should be estimated and accounted for. Interestingly, the most well known socio-demographic predictors of patient satisfaction (age and self-perceived health)1 were not associated with overall patient satisfaction in our study. This indicates that the main differences between these groups are related to expectations, and that socio-demographic variables might be redundant in studies that include questions on patient expectations.
The regression model in the present study explained almost 60% of the variation in overall patient satisfaction. This explanatory power is high compared with other studies,12–14 but one study found an even higher explained variance.15 The level of explained variance indicates that the most relevant sources of variation in patient satisfaction—as measured in the present questionnaire—were included, which concurs with a holistic approach to patient satisfaction incorporating patient-reported experiences, patient expectations and socio-demographic variables as predictors. The unexplained variance seems to be mostly related to measurement errors. The test–retest reliability of the patient-satisfaction item was 0.73, meaning that an estimated 27% of the variance was explained by measurement errors. Consequently, potential predictors not included in the model seem to have limited impact.
The primary purpose of national patient-experience surveys in Norway is related to quality improvement, healthcare management, public accountability and patient choice of hospitals. Results from the inpatient survey in 2006 were aggregated at the hospital level and case-mix adjusted in hospital comparisons. The case-mix model included age, gender, self-perceived health, education, admission type and Charlson comorbidity index.9 These factors are not controllable for hospitals, and so fair comparisons require such adjustments. The effect of case-mix adjustment was relatively small for all patient-reported experience scales, which is in line with other findings.16 17 We believe that a large part of the unexplained variance reflects differences in patient experiences.18 However, the observed correlations between patient-reported experiences and fulfilment of expectations in our study mean that expectations could also be an important predictor. Regression analysis with patient-reported experience scales as dependent variables, and demographic variables and fulfilment of expectations as independent variables showed that fulfilment of expectations is the most important predictor (results not presented here). Patient-reported fulfilment of expectations is a result of the assessment of prior expectations versus current experiences with the hospital. Prior expectations can only be partly controlled by single healthcare providers, and hence hospital experiences of equal quality may be rated differently solely due to differences in prior expectations. Consequently, using fulfilment of expectations as a quality indicator would mean that poorly performing hospitals could be ranked highly if the patients had low prior expectations on average. Similarly, using fulfilment of expectations in case-mix models for patient-reported experiences would include components amendable by hospitals, namely patient experiences. Therefore, methods to reduce the effect of expectations on patient rating are needed, which is one important argument in support of the patient-reported experience approach.19–21 However, this approach was used in the present study, indicating that the measurement of prior expectations, their sources and the possibility of including expectations in case-mix adjustments should be studied in future research.
Limitations
One limitation of this study was the use of single items about expectations and overall patient satisfaction. For example, the importance of expectations regarding certain aspects of quality might vary between patients, and therefore exert different effects on patient satisfaction. This was demonstrated in a recent study that found that patients' priorities varied between patient groups and the overall satisfaction score was more influenced by their experiences on priority aspects.22 Patient-satisfaction questions could also have been formulated for all important aspects of the healthcare encounter.15 Therefore, a more precise operationalisation of both patient expectations and patient satisfaction would have been achieved by using multi-item scales. This was not possible in the present study, since the single items about expectations and satisfaction constituted a minor part of a measurement instrument for patient-reported experiences. Although the items about overall satisfaction and overall assessment of the hospital versus expectations were above the commonly used reliability criterion of 0.7 for ICC,23 future research on this topic should include psychometrically validated instruments for the three central concepts.
A second potential limitation of the current study is that all patient-reported variables were completed after discharge, creating a potential ‘response-shift’ problem, which refers to a change in the meaning of one's self-evaluation of a target construct due to changes in internal standards, values or definition of a concept.24 In this context, patient expectations might change as a result of experiences during the hospital stay, such as through a conscious or unconscious adjustment of expectations to perceived experiences. Therefore, it is difficult to argue that the expectations measured in the present study adequately reflected patient expectations before their admission to hospital. A longitudinal approach could have been better, such as by measuring expectations before admission, patient-reported experiences following discharge and patient satisfaction some time after discharge. It is important to note that our approach to measuring expectations is a comparison of perceived experiences to patient expectations, (ie, fulfilment of expectations defined as the hospital assessment being better than expected, as expected, or worse than expected). A linear partial effect of patient fulfilment of expectations on overall patient satisfaction was assumed, going from ‘much poorer than expected’ to ‘much better than expected’. The response-shift effect constitutes a measurement problem for all patients comparing their experiences and expectations. However, we were not primarily interested in estimates and explanations for patient expectations, but rather in assessing the partial effect of fulfilment of expectations on overall patient satisfaction. Regardless of changes in expectations from prior admission to post-discharge, our study shows that patient evaluation of the gap between expectations and experiences is a powerful predictor of patient satisfaction. When all else is equal, perceived experiences that are worse than expectations predict less-satisfied patients, while perceived experiences that are better than expectations predict more-satisfied patients. This is in line with the confirmation of expectation paradigm.6 Nevertheless, there is no consensus regarding how to operationalise and measure patient expectations.5 Various approaches can be applied, such as those that define and measure expectations as independent needs or priorities.25
A third potential limitation of the current study is the response rate. In general, postal surveys have lower response rates than other data-collection modes.1 The response rate in the current study (46%) is average for Norwegian national patient-experience surveys. Non-response bias occurs when the main variables differ systematically between respondents and non-respondents.26 The findings from a Norwegian follow-up study involving a similar inpatient population showed that postal respondents and non-respondents had almost the same patient-reported experience scores.27 This is in line with the results of other Norwegian user-experiences studies,28 29 indicating that the response rate exerts only minor effects.
Conclusions
The study showed that both fulfilment of expectations and patient-reported experiences are distinct from but related to overall patient satisfaction. The most important predictors for overall patient satisfaction with hospitals are patient-reported experiences and fulfilment of expectations. Future studies should test methods designed to measure prior expectations, including sources of influences on expectations.
Acknowledgments
We thank Saga Høgheim, Reidun Skårerhøgda and Kari Aanjesen Dahle for their contribution to the national data collection, and Tomislav Dimoski for developing the FS-system and carrying out the technical aspects of the national survey.
References
Footnotes
Competing interests None.
Ethics approval Ethics approval was provided by The Norwegian Regional Committee for Medical Research Ethics, the Data Inspectorate and the Norwegian Directorate of Health and Social Affairs approved the survey.
Provenance and peer review Not commissioned; externally peer reviewed.