Identifying patients at risk and ensuring they are on appropriate therapy is the mainstay of preventive medicine and an integral role of the GP. The process of audit can highlight areas of suboptimal management, and in recent years electronic audit packages have become increasingly popular.
THE USE OF AUDIT SOFTWARE IN GENERAL PRACTICE
PRIMIS (Primary Care Information Services), a group from the University of Nottingham, currently have 21 different audit tools available, ranging from asthma care to flu jab uptake, to oral contraception.1 This is the organisation that created the GRASP-AF audit tool in collaboration with the West Yorkshire Cardiovascular Network. This software interrogates primary care databases with a predefined set of search criteria and therefore relies on the accuracy of data recording and coding performed at the practice.
We investigated the management of atrial fibrillation (AF) in January 2014 with anticoagulation for effective stroke prevention in four general practices in the West Midlands. AF, being the most common sustained cardiac arrhythmia, should be one of the areas in which patient management is at its best.2 This is particularly important as a patient with AF is five times more likely to experience stroke.3 It has been consistently reported that there has been under-treatment with oral anticoagulation.4–6
At the outset we planned to use the GRASP-AF toolkit and assess against the 2006 NICE guidelines, which were the most current at the time. Patients at high risk were categorised by having a CHADS2 score ≥2, and these patients should have been prescribed or considered for anticoagulation.7 In the first practice that participated, the GRASP-AF toolkit indicated that 24.39% (20/82) of patients with AF at high risk of stroke were not anticoagulated and had no recorded contraindication or treatment refusal.
When these data were fed back to the practice, there was surprise and we were urged to re-audit the practice manually.
MANUAL VERSUS AUTOMATED AUDIT TOOLS
The manual audit was conducted by searching the GP database for all patients with unresolved AF. A CHADS2 score was calculated for each patient and the notes were then scrutinised to see if they were receiving antithrombotic therapy. If there was no anticoagulation prescribed, notes were searched to identify a reason: a contraindication or refusal. When manual audit was complete, it was found that just one patient had no documented reason for lack of anticoagulation (1/78, 1.28%). Four records included in the original analysis were not available for full manual audit.
This discrepancy led to reflection on the use of the GRASP-AF toolkit. Installation was found to be complicated and time consuming, and raised concerns about data security despite reassurance. The process required a moderate level of computer expertise and we felt that available guidance was suboptimal.
Discrepancy between GRASP-AF output and manual audit appeared related to the fact that the practice had a high number of patients who were not anticoagulated but had a recorded contraindication. Furthermore, doctors often used free text rather than predefined Read Codes, which may explain why the software did not identify that these patients had a recorded refusal or contraindication. It is of course entirely possible that contraindications noted may represent clinician perception and be subject to individual bias; for example, exaggerated concern over risk of falls when it has been suggested that a patient would have to fall 295 times before the risk of falls outweighs the benefit of anticoagulation.8
In this context the GRASP-AF output may be more clinically meaningful and highlight the need for further consideration; however, the fact remains that it is intended to function as an accurate audit tool and in this context we did not find this to be the case.
Similar issues have been emphasised in other areas in relation to electronically-derived audit data. The need for consistent coding in diabetes has been highlighted9 and a recent study of depression management stressed that duration of treatment was associated with the proportion of patients being coded as having depression,10 underlining the impact of coding on individual patient care.
Furthermore, studies that use electronic audit to explore disease prevalence and conclude that results reflect underdiagnosis, such as a study of occupational asthma,11 may be subject to biases introduced by coding being poorly aligned to audit objective.
IMPACT ON CLINICAL PRACTICE
This single-site comparison of electronic and manual audit shows that this automated audit tool did not accurately classify patients who may be being suboptimally managed, and our concern is that this problem is having widespread impact on audit and research. Going forward there are three options: GPs need to improve their consistent use of predefined Read Codes; audits should continue to be done manually; or software should be more rigorously tested to ensure full compatibility with existing records. It is this last point we wish to stress: data obtained using the GRASP-AF toolkit have been previously published12,13 and will be influencing practice. Validation of this and similar tools in real-life scenarios in the way we have demonstrated is critical to ensure that data obtained from large-scale studies are accurate and trustworthy.
- © British Journal of General Practice 2015