INTRODUCTION
Back in the 1990s, Oxman et al concluded that ‘there are no “magic bullets” for improving the quality of health care’.1 Today, almost 30 years later, the conclusion is still the same, despite a plethora of studies having evaluated the effectiveness of strategies to change healthcare professionals’ behaviour and improve patient care. In general, passive dissemination strategies such as the distribution of educational materials appear largely ineffective, while interventions based on action, such as audit and feedback, educational meetings, educational outreach visits, and reminders, have been shown to be more effective.2,3
Interventions more likely to be successful seem to act through the Normalisation Process Theory constructs that explain implementation mechanisms: coherence (sense making of interventions); cognitive participation (engagement with intervention); collective actions (work done to enable intervention to happen); and reflexive monitoring (cost–benefit appraisal).4 We hereby present the Audit Project Odense (APO) method, which seeks to address all the dimensions of the Normalisation Process Theory by self-registration and open discussion of the identified behaviour.
HISTORY OF THE APO METHOD
In the 1970s, a simple chart was developed at the Birmingham Research Unit for General Practice, suitable for prospective self-registration of activities in general practice.5 In England many different practice activities were registered by means of this chart, but when the registrations were repeated 1 year later hardly any changes had taken place. In the late 1980s, four Danish GPs from the Department of General Practice at Odense University visited the Birmingham department and were taught the basic rules for conducting these practice activity analyses. The APO team refined the English chart and in addition a thorough course activity was offered to participants between the two rounds of registration. In 1989, the APO unit was established and 4 years later it became part of the Research Unit for General Practice at the University of Southern Denmark.6 The first application of the APO method on acute respiratory tract infections in 1992–1993 obtained significant intervention results, thereby increasing the interest in the method.7 Since then, the APO method has been used for hundreds of projects targeting multiple topics, settings, and countries.
CHARACTERISTICS OF THE APO METHOD
Table 1 provides an overview of the general rules that apply to projects using the APO method. Topics suitable for the method occur frequently — preferably at least 30 times during a 4-week period. Audits on frequent topics can be performed in just 1 week or even a single day; however, most audits do proceed for 3–4 weeks. In general, the more patient contacts registered, the better the results, as many registrations increase the preciseness and allow a higher level of detail.
Table 1. Basic rules for data registration by means of the Audit Project Odense method
The registrations are performed on A4-size paper charts with APO’s specific layout. The paper instrument is simple, transparent, and easy to transfer between topics and settings. So far, only two projects have offered healthcare professionals the choice between registering electronically or on paper. Simplicity is fundamental since clinical settings usually deal with busy agendas. Typically, a well-designed APO chart can sufficiently uncover a topic via just five to 10 ticks per patient contact. Filling in the information required for one patient preferably takes less than 1 minute.
After the registration period, all the information collected is compiled in a report. This report summarises information about patient characteristics (such as age and sex), symptoms, clinical findings and examinations, diagnoses, and choices of treatment. The variation in performance between the participating healthcare professionals is reported anonymously via ID-number in the various diagrams. In addition, each participant receives individual feedback on their own performance.
The results of the audit are conveyed at a follow-up meeting where all participating healthcare professionals, the project team, and experts in the audited topic participate. The overall aim of this meeting is to uncover and evaluate potential quality problems in the healthcare professionals’ management of the topic being investigated. The APO quality circle usually includes a second final registration — about a year after the first registration — to evaluate to what extent the identified quality problems have been solved.
STRENGTHS AND LIMITATIONS
Munck et al have demonstrated that data registered by means of the APO method are reliable and practically identical to information collected in the medical records.8 Although APO data have proven valid, findings should be interpreted with caution. Perhaps the most important limitation of data collected by means of the APO method is the lack of external validity. Voluntary participation may reduce generalisability. For example, Strandberg et al have found that GPs signing up for an audit tend to have a more rational use of antibiotics than non-participating GPs.9 Also, the Hawthorne effect has to be taken into account as healthcare professionals might change their behaviour when they know they are being observed.
Most evidence on the effectiveness of the APO methodology is from prospective before-and-after studies. This design has always been considered as a drawback hampering publication in prestigious journals. However, a few randomised clinical trials have confirmed the effect of the APO methodology.10,11
Although data collected by means of the APO method are less suitable for estimating disease prevalence, several associations identified in APO data are likely to be generalisable, for example, associations between patient or healthcare characteristics and various treatment regimens.12
The cross-sectional nature of the APO method is another weakness, as data only reflect ‘a snapshot of the real world’. Variables included in the registration chart are lined up as expected following the consultation process. Theoretically, decisions on treatment(s) are taken after a diagnosis has been established. However, GPs may decide on treatment at the same time as, or even before, making the diagnosis. Afterwards, the GP adjusts the interpretation of findings and makes the diagnosis fit the treatment decision. This may lead to a diagnostic misclassification bias.
ETHICS
Written informed consent is required from healthcare professionals signing up for an APO audit. Participants agree that both information about themselves, such as age, sex, and seniority, and activity data (registrations) are used for both the quality improvement project and appertaining research. Importantly, data are pseudonymised before being used for any research activities and individual participants cannot be identified in publications. The method does not allow time for obtaining informed consent from patients. Thus, it is only permitted to include data in the APO registration chart in which individual patients cannot be identified.
PERSPECTIVES
The APO method has proven effective in improving the performance of healthcare professionals. However, a process evaluation would be valuable to obtain more detailed information about how and why the method works, and to generate information about how to improve the method.
So far, most projects applying the APO method have been conducted in the general practice setting and often only involving GPs. However, several projects have also engaged other healthcare professionals, such as ear–nose–throat specialists, physiotherapists, chiropractors, and practice staff, including nurses. Hopefully, the future will bring more projects involving various types of healthcare professionals and different settings, such as hospital departments, nursing homes, pharmacies, and dental clinics.
Notes
Provenance
Freely submitted; externally peer reviewed.
Competing interests
Carl Llor reports having received research grants from Abbott Diagnostics. All other authors have declared no competing interests.
- © British Journal of General Practice 2022
