Abstract
Aim To study the effects of a team-based model for continuous quality improvement (CQI) on primary care practice management.
Design of study Randomised controlled trial.
Setting Twenty-six intervention and 23 control primary care practices in the Netherlands.
Method Practices interested in taking part in the CQI project were, after assessment of their practice organisation, randomly assigned to the intervention or control groups. During a total of five meetings, a facilitator helped the teams in the intervention group select suitable topics for quality improvement and follow a structured approach to achieve improvement objectives. Checklists completed by an outreach visitor, questionnaires for the GPs, staff and patients were used to assemble data on the number and quality of improvement activities undertaken and on practice management prior to the start of the intervention and 1 year later.
Results Pre-test and post-test data were compared for the 26 intervention and 23 control practices. A significant intervention effect was found for the number of improvement objectives actually defined (93 versus 54, P<0.001) and successfully completed (80 versus 69% of the projects, P<0.001). The intervention group also improved on more aspects of practice management, as measured by our practice visit method, than the control group but none of these differences proved statistically significant.
Conclusion The intervention exerted a significant effect on the number and quality of improvement projects undertaken and self-defined objectives met. Failure of the effects of the intervention on the other dimensions of practice management to achieve significance may be due to the topics selected for some of the improvement projects being only partly covered by the assessment instrument.
INTRODUCTION
Optimal practice management is a prerequisite for good clinical care.1,2 A well-managed practice is the showpiece of a practice, and patients have been shown to strongly value these service aspects of a practice.3,4
In Australia, New Zealand, the US and Europe, instruments have been developed to assess practice management.5 For instance, in the UK, approximately 20% of the indicators within the Quality and Outcomes Framework of the new GMS contract concern the organisational aspects of care.6 In the Netherlands, assessment of the practice organisation is gaining importance. The Visitation Instrument for Practice (VIP) management has already been implemented on a widespread basis in the Netherlands, and an essential part of a practice accreditation procedure.7 A set of management indicators and an assessment instrument have also been recently developed to compare practice management across countries at a European level.8 However, the assessment of practice performance and the provision of feedback has often been found to not be enough to improve the management of primary care services.9 Concrete support appears to be necessary to help practices modify their performance. An intervention aimed to improve the management of primary care after initial assessment and feedback has therefore recently been developed by the present authors in the Netherlands. The principles underlying the intervention have been derived from models of continuous quality improvement (CQI). A bottom-up approach to CQI stands central along with an active role for the practice team and the application of a clearly structured, stepwise problem-solving method to develop and implement the improvement plans.10–13 Outreach visitors (facilitators) are used to coach and train the team on the spot, as such educational outreach efforts have been shown to be a promising approach for the promotion of preventive care efforts and for more critical prescription practices among GPs.9,14,15
How this fits in
Continuous quality improvement has been described and used as a strategy for quality improvement in general practice, but evidence for the effectiveness is largely lacking. To fill this gap, an randomised controlled trial was undertaken and described in this paper.
Although such a team-based CQI approach to quality improvement has been applied to primary care services before,16–19 clear evidence for the effectiveness of such an approach is still largely lacking.20 To fill this gap, a randomised controlled trial was therefore undertaken to answer the following research question — What is the effect of a team-based CQI intervention on the number and quality of the improvement projects undertaken and the management of primary care?
METHOD
Design
A randomised controlled pre-test/post-test design was used. Between October 2001 and April 2003, 215 primary care practices that were on a list for a practice assessment using the national Dutch VIP management were invited by the outreach visitor during the feedback session to participate in the present study (Box 1). They were given a letter of invitation which indicated that participation in the study was strictly voluntary and that confidentiality would be guaranteed.
Box 1. Core elements of the team-based CQI model.
▸ Assessment of current practice management using the Visitation Instrument Practice management (VIP)7
The content of the VIP was established in 1995 by having GPs and experts define the domain of practice management and obtain consensus on a number of quality indicators that are also regularly updated. The VIP assesses practice management at the level of the practice and the level of the GP. The VIP consists of a half-day practice observation by an outreach visitor during normal practice hours and the administration of questionnaires to each GP, one practice nurse and 30 patients (to be filled in while waiting in the waiting room) per practice, of which at least 22 need to be filled in and returned.
▸ Detailed oral and written feedback
Benchmark figures and suggestions for how to improve the weak aspects of practice management are provided after the VIP visit.
▸ CQI tools
A workbook with an explanation of the CQI model, a description of the steps to be taken and checklists and forms for easy project registration.
▸ A trained facilitator
An outreach visitor provides CQI guidance and support during five monthly team meetings. The weak aspects of the team's practice management are discussed; help is provided with the selection of suitable topics for improvement; 21 and the team is guided through the quality improvement cycle.
▸ Quality improvement cycle. Both during and between the monthly meetings, the following quality improvement cycle is followed
Measurement and description of the present situation, formulation of a target, identification of potential barriers, determination and implementation of change and improvement activities, observation of progress and outcome evaluation.
▸ Continued application of the model
The facilitator gradually transfers various tasks to an interested member of the team. In between the third and fourth outreach visits and in between the fourth and fifth outreach visits, the practices also meet without the facilitator to further customise their work.
Those practices that agreed to participate were, after stratification for region and group size, randomly allocated to the intervention or control conditions by a minimisation programme. An independent statistician with the aid of a computer program carried out the randomisation.
Intervention
Assessment of the practice management using VIP was the starting point for the intervention (box 1).7 After initial assessment, the practices in the intervention group undertook a CQI process with the help of an outreach visitor. The outreach visitors were all experienced practice assistants who had also participated in a 3-day training programme to learn how to organise the CQI meetings, guide the practice team through the steps of the CQI model and deal with group processes in general. The core elements of the team-based CQI model are described in Box 1.
The practices in the control condition were only provided the written feedback from the VIP and related suggestions for improvement during the usual 1-hour meeting that occurs as part of the VIP.
Data collection
The following instruments were used to study the effects of the CQI intervention.
Written questionnaires with return envelopes were sent to the principal GP of each participating practice at the beginning of the study with open questions regarding improvement activities undertaken over the past year. The GPs were also asked to indicate the different steps taken per improvement project using a list with tick boxes (baseline measurement, formulation of targets, identification of barriers, development and implementation of change plan, measurement and description of results, meeting of self-defined objectives). The same written questionnaire was administered 1 year later.
VIP assessments before and after intervention were used to determine the effects of the team-based CQI intervention on predefined practice management dimensions. The VIP includes over 50 dimensions, but we restricted the analyses to the 20 dimensions that were related to topics chosen in intervention or control group (Table 1).
Effects of the team-based CQI intervention on VIP dimensions of practice management (scale scores have been converted into percentage of maximum score; minutes are raw means).
Analyses
The numbers of improvement projects undertaken by the practices in the year before the intervention period and during the intervention period were calculated as well as the number of improvement steps taken and the number of self-defined objectives met. In a linear mixed regression model with compound symmetric structure for repeated measures and all variables treated as fixed effects, the numbers of improvement projects undertaken by the intervention and control groups after correction for the numbers of projects undertaken in the year before the intervention period were compared. Using mixed logistic regression analyses, the improvement steps taken by the intervention and control groups were compared after correction for the steps taken in the year before the intervention period.
Finally, an overview of the improvement projects formulated by the intervention and control groups was created, and we tried to link appropriate predefined dimensions of practice management from the VIP to the improvement projects. To test for an effect of the team-based CQI intervention on overall practice management, univariate regression analyses were performed to compare the intervention and control groups with the post-test scores for the predefined dimensions of practice management from the VIP serving as the dependant variables and the pre-test scores for the same dimensions serving as the independent variables. We used SPSS 12.0.1 software and for the mixed logistic regression SAS V8.2.
RESULTS
Of the 215 practices on the waiting list for a VIP assessment, 49 (23%) agreed to participate in the present study. After matching according to practice size, 26 practices were allocated to the intervention condition (six single-handed and 20 larger practices) and 23 practices were allocated to the control condition (six single-handed and 17 larger practices). Although all intervention practices completed the intervention, a total of two intervention practices (one due to illness and one due to a reported lack of interest on the part of the team members) and two control practices (one due to moving the practice and one due to lack of time) did not complete all the questionnaires, which meant that we completed post-test measurements of 24 intervention and 21 control practices. All interventions took place between December 2001 and February 2004.
Number and quality of improvement projects undertaken
In the year before the intervention period, the practices in the control group undertook an average of 2.4 improvement projects and the practices in the intervention group undertook an average of 1.5 projects. The practices in the control group also met their self-defined objectives more often at pre-test than the practices in the intervention group (Table 2). After correction for these initial differences, the practices in the intervention group can be seen to initiate significantly more improvement projects during the intervention period than the practices in the control group with means per practice of 3.9 and 2.6, respectively. All of the steps needed for effective quality improvement were performed for a significantly larger percentage of the projects undertaken by the intervention group than by the control group with the exception of the identification of barriers (Table 2). The practices in the intervention group met the self-defined objectives for 80% of their projects; the practices in the control group did this for 69% of their projects.
Overview of projects following different steps for improved management of primary care services (percentage of initiated projects).
Changes in dimensions of practice management
In sum, the intervention practices improved on 15 dimensions of practice management and declined on four dimensions. The control practices improved on 11 dimensions and declined on nine dimensions.
In both the intervention and control groups, the practices frequently selected accessibility and availability of practice services, which includes access by phone and organisation of the appointment system, as a topic for improvement: this was done on 30 occasions by the intervention practices and on 17 occasions by the control practices during the intervention period. The patients of the intervention practices reported being as satisfied with the accessibility and availability of the practice at post-test as at pre-test, as well as the patients of the control practices (Table 1). Also the time before the practice picks up the phone remained unchanged in both research groups.
Other favourite topics selected for improvement were ‘medical care’ topics, such as the chronic disease management and preventive care (17 and 11 times by the intervention and control groups, respectively, during the intervention period). Given that the delivery of chronic disease management is a relatively new development, there was no separate measurement of this specific aspect of the management of primary care services available for analysis. Nevertheless, the supply of preventive care was found to be about the same for the intervention and control groups at pre-test and to also increase more or less equally for the two groups at post-test. The delegation of health promotion tasks, including diabetes care, increased from 39 to 46% for the intervention group and stayed the same (38 and 39%) in the control group.
Topics related to ‘infrastructure’ were chosen 13 times in the intervention group and nine times in the control group. Of the VIP dimensions that more or less covered these topics, the ‘hygiene and facilities in the treatment room’ improved slightly from 55 to 63% for the intervention group and stayed the same in the control group (60 and 58%, respectively).
Regular meetings with the practice team was selected as a topic for improvement by several of the practices (eight and five times by the intervention and control groups, respectively). The GP meeting time with practice assistants did not change in the intervention group but decreased by some 16 minutes in the control group. Similarly, the weekly meeting time with colleagues in the intervention group remained about the same (42 minutes at pre-test and 46 minutes at post-test) as well as in the control group (38 and 35 minutes, respectively).
A total of seven practices in the intervention group and three practices in the control group selected improved medical registration (that is, computer registration) as a topic for improvement. In the intervention group, this dimension of practice management was found to shift from 56 to 67% while no change was observed in the control group. Although improved communication with hospitals and pharmacies was not selected as an actual topic for improvement by any of the practices examined in our study, both the intervention and control practices showed improved electronic communication with hospitals and pharmacies during the intervention period.
None of the preceding differences between the intervention and control groups after control for the pre-test values were found to be statistically significant (Table 1).
DISCUSSION
Summary of main findings
The practices exposed to our team-based CQI intervention within the context of the present study initiated and completed significantly more quality improvement projects than the practices in the control group. The projects ran in a more structured way and in significantly more projects the self-set goals were met. As practices were free to select their own topics for improvement and set their own objectives, the fact that the intervention group met a significantly greater number of self-defined improvement objectives than the control group is an important finding.
In terms of the VIP dimensions the intervention practices were also more successful, but the differences between intervention and control practices were small and non-significant.
Strengths and limitations of the study
The results of this carefully randomised study show a modest but undeniable contribution of the team-based CQI intervention to a culture of improvement. We performed the first randomised controlled trial on continuous quality improvement in general practice. The VIP instrument, which was used to provide the practices with an extensive overview of their weak and strong practice management aspects as well as an outcome measurement, has shown to be able to discriminate between practices22 and to show trends in time.23–25
Some possible limitations on the present study are as follows. To start with, both the intervention and control groups in our study consisted of practices which volunteered to participate and therefore practices that were probably very motivated to change. A motivated practice appears to be a prerequisite for successful improvement and the effective implementation of improvement efforts. Use of the team-based CQI intervention examined here with other, less motivated, practices may therefore require more extensive support and perhaps financial or other incentives.
Second, although we performed a careful randomisation of the practices that were interested with the help of a computer program and an independent researcher, the control practices were doing better regarding CQI at the beginning of the study than the intervention practices. We do not know whether a part of the improvement in the intervention practices is due to ‘regression to the mean’.
Third, as a consequence of the fact that practices could freely select the topics for organisational change, the number of practices for a particular topic was in many cases quite small (that is, the number of practices for a particular change topic was only a fraction of the total number of practices) and thus too small to produce statistically significant change at the level of the group. Besides, although the practices made use of the VIP feedback and indicators to select topics for improvement, the improvement objectives formulated by the practice teams did not have to match the VIP indicators, which may have obscured many intervention effects. As the practices were free to define their own improvement objectives, we could not know — a priori — just which aspects of practice management might improve. As a result of this situation, different aspects of the intervention practices may have improved than measured by the VIP and a significant effect in terms of the VIP therefore not detected. For example, the VIP does not measure dimensions of practice management related to staff workload, and reducing staff workload was often the main objective when practices selected practice accessibility and availability as starting point and improvement objective.26
In fact, access by phone was in some practices limited as a consequence of more structured working day for the practice nurses. As a consequence of this free topic choice and therefore a multifactorial design, a power analysis was complicated as it was very difficult to predict which topics would be chosen. Despite this uncertainty and the lack of figures from similar studies, we did perform a power calculation before we began our study. To find a difference of 25% in outcome between intervention (75%) and control group (50%), with a 5% two-sided significance level and a power of 80%, we needed 55 practices per group. Although 25% of the practices that were invited to take part agreed, which is much more than in similar studies,16,17 we were not able to include more than 49 practices in total. The most important barrier for practices to take part in our study was the randomisation: very many practices were enthusiastic about the intervention, but they did not accept the chance to be allocated to the control group and therefore decided not to take part in the study.
Finally, the present study was undertaken in a period when practice nurses were first being introduced into Dutch primary care. All of the practices in our study were preparing for the introduction of such a new professional or had recently appointed a practice nurse. The introduction of such a new function certainly cost both the intervention and control practices considerable time and energy. Alternatively, the introduction of such a new function may have motivated both the intervention and the control practices to examine their management and adjust this as needed. Relative to the control practices, thus, the intervention practices may not have changed enough and thus as a result of our intervention, to produce statistically significant differences in the management of primary care services. Yet, general practice is always in a state of change so any intervention has to show its worth in that context.
Implications for clinical practice and future research
The team-based CQI model will be offered to practices in the Netherlands as one of the support tools in the accreditation procedure. During the first accreditation assessment, practices need to define improvement goals, which will be evaluated during a reassessment after 3 years. As this provides a new incentive to improve, the effect of this should be evaluated. Besides, as many practices in our project chose improvement topics related to staff workload, we recommend that in future evaluations workload of staff and the team climate inventory (TCI) will be included.27
Acknowledgments
We would like to thank the general practices that participated in the project.
Notes
Funding body
The Netherlands Organisation for Health Research and Development ((ZonMw) (1615.0004)
Ethics committee
Not applicable
Competing interests
The authors have stated that there are none
- Received February 25, 2005.
- Revision received October 18, 2005.
- Accepted March 13, 2006.
- © British Journal of General Practice, 2006.
REFERENCES
In this issue
Jump to section
Keywords
More in this TOC Section
Related Articles
Cited By...
