Abstract
Background The diagnosis of cancer in primary care is complex and challenging. Electronic clinical decision support tools (eCDSTs) have been proposed as an approach to improve GP decision making, but no systematic review has examined their role in cancer diagnosis.
Aim To investigate whether eCDSTs improve diagnostic decision making for cancer in primary care and to determine which elements influence successful implementation.
Design and setting A systematic review of relevant studies conducted worldwide and published in English between 1 January 1998 and 31 December 2018.
Method Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials were searched, and a consultation of reference lists and citation tracking was carried out. Exclusion criteria included the absence of eCDSTs used in asymptomatic populations, and studies that did not involve support delivered to the GP. The most relevant Joanna Briggs Institute Critical Appraisal Checklists were applied according to study design of the included paper.
Results Of the nine studies included, three showed improvements in decision making for cancer diagnosis, three demonstrated positive effects on secondary clinical or health service outcomes such as prescribing, quality of referrals, or cost-effectiveness, and one study found a reduction in time to cancer diagnosis. Barriers to implementation included trust, the compatibility of eCDST recommendations with the GP’s role as a gatekeeper, and impact on workflow.
Conclusion eCDSTs have the capacity to improve decision making for a cancer diagnosis, but the optimal mode of delivery remains unclear. Although such tools could assist GPs in the future, further well-designed trials of all eCDSTs are needed to determine their cost-effectiveness and the most appropriate implementation methods.
INTRODUCTION
A timely diagnosis of cancer is critical, as delays are associated with poorer patient outcomes and survival rates.1,2 GPs play a key role in early cancer diagnosis, with 75–85% of cases first presenting symptomatically in primary care.3–5
The primary care interval describes the time from first symptomatic presentation to the GP, through to referral to a specialist.6 The length of this interval varies, with many patients presenting to their GP three or more times before referral.7 Consequently, interventions that assist GPs’ clinical decision making have the potential to improve the timeliness of cancer diagnosis and improve cancer outcomes.
Electronic clinical decision support tools (eCDSTs) are electronic systems that assist clinical decision making.8 Patient-specific information is entered into the eCDST by the GP or can be automatically populated from the patient’s electronic health record. Using validated algorithms, the eCDST produces recommendations, prompts, or alerts for the GP to consider. eCDSTs can be actively used during a GP consultation or may be designed to continuously mine data in the background.
The development of eCDSTs has been driven by the complex nature of a cancer diagnosis. Often, patients present to the GP with non-specific symptoms that have a low diagnostic value.9 Algorithms have been designed to apply epidemiological data on combinations of symptoms and test results, and prompt consideration of a cancer diagnosis based on cancer risk thresholds.10,11 eCDSTs have been proposed as a solution for cancers that are more challenging to diagnose in primary care because of their variable symptomatic presentation and limited specific features.12
eCDSTs have been shown to improve both practitioner performance11 and diagnostic accuracy in simulated patients for a range of conditions, such as dementia, osteoporosis, and HIV.13 The effects of eCDSTs on referral behaviours have been summarised in a previous systematic review,14 but it did not investigate cancer diagnosis specifically; consequently, the role of eCDSTs in cancer diagnosis has not been adequately addressed.
This systematic review aimed to summarise existing evidence on the effects of eCDSTs on decision making for cancer diagnosis in primary care, and determine factors that influence their successful implementation.
Electronic clinical decision support tools (eCDSTs) improve practitioner performance and patient care, but their role in cancer diagnosis has not been adequately addressed. This review outlines the effectiveness of eCDSTs for cancer diagnosis and factors affecting their implementation. Decision support tools have been proposed as an approach to reduce delays in diagnosis, particularly for cancer with non specific symptom signatures. To the authors’ knowledge, this is the first systematic review of available publications to inform eCDST implementation in primary care for the diagnosis of cancer. |
METHOD
A mixed-methods narrative review was conducted. The review was registered on PROSPERO (registration ID: CRD42018107219) and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria were followed.15
Search strategy
Electronic searches were run across three databases: MEDLINE, EMBASE (Ovid), and the Cochrane Central Register of Controlled Trials (CENTRAL). The search strategy (available from the authors on request) included MeSH headings and word variations for three terms: ‘general practitioner’, ‘cancer’, and ‘electronic decision support’. All studies from 1 January 1998 until 31 December 2018 were included. Titles and abstracts were screened independently by two reviewers and any disagreements were resolved with a third researcher. To identify studies not found via the electronic searches, reference lists were manually checked, citation tracking was performed, and experts in the field were contacted. The corresponding authors from all the included studies were contacted via email to identify further studies or unpublished research.
Inclusion and exclusion criteria
Studies investigating an eCDST designed to aid decision making for a potential cancer diagnosis were selected. For inclusion in the review, the study had to report on a:
cancer diagnosis;
cancer referral; or
cancer investigation.
Healthcare utilisation and cost, practitioner performance, and other educational outcomes were also included in the study. As the mode and delivery of eCDSTs vary, studies using any form of electronic support that included algorithm-based prompts or recommendations were eligible. Tools that applied risk markers for prevalent undiagnosed cancer were included, as were qualitative studies if they evaluated barriers and facilitators to implementing eCDSTs for cancer diagnoses in primary care.
Exclusion criteria for qualitative and quantitative studies included:
decision support used for cancer screening in asymptomatic populations, including tools that incorporated risk factors to predict future incident risk of cancer;
studies that did not involve decision support designed for use in primary care;
articles not in English;
unpublished work;
editorials; and
academic theses.
Assessment of bias
Several Joanna Briggs Institute (JBI) Critical Appraisal Checklists were used, depending on the study design of the articles included in the systematic review, to assess the risk of bias of included studies.16 The authors used the following JBI Critical Appraisal Checklists:
Checklist for Quasi-Experimental Studies (non-randomised experimental studies);
Checklist for Qualitative Research;
Checklist for Diagnostic Test Accuracy Studies; and
Checklist for Randomized Controlled Trials.
Studies with a percentage score of >80% were considered to have a low risk of bias; those with a percentage score of 60–80% were considered to have a moderate risk of bias.
Study design
Data were extracted and analysed separately from included studies, before all results were combined into an extensive narrative synthesis. Segregated methodology was used to synthesise the evidence while maintaining the standard distinction between quantitative and qualitative research, in line with recommendations.17,18
Data extraction and synthesis
Data extraction was performed, and cross-checked by the two reviewers who screened the articles. For quantitative studies, data extraction was based on an adapted version of the Cochrane Effective Practice and Organisation of Care data-collection checklist.19 For qualitative studies, data extraction was performed by a reviewer who screened the original articles, together with a third author, guided by the approach of Noblit and Hare.20 This involved identifying the major themes from primary papers, determining how they are related, and building on themes to interpret overarching theories and understandings.20
Categorisation of extracted themes was based on the normalisation process theory (NPT) framework,21 a theory used to describe factors and actions that promote or impede the embedding of new technologies into an existing practice. NPT uses four constructs to explain the processes that affect the integration and adoption of new technologies:
Using these constructs, the barriers and facilitators identified in the qualitative studies were mapped onto the NPT framework to explain the results.20
RESULTS
In total, 1065 titles were identified and 66 full-text papers reviewed for eligibility (Figure 1). Twelve articles, reporting on nine individual studies, fulfilled the selection criteria: eight quantitative,24–31 three qualitative,32–34 and one mixed methods.35 Characteristics of the included studies are summarised in Table 1.
Figure 1. PRISMA flow diagram of literature search. eCDST = clinical decision support tool. PRISMA = Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Table 1. Characteristics of included studies
The design of each eCDST and key results are summarised in Table 2. Outcomes included:
appropriateness of care (n = 5);
diagnostic accuracy (n = 1);
time to diagnosis (n = 1);
cost-effectiveness (n = 1);
process measures (n = 1); and
qualitative (n = 4).
Table 2. Quantitative study descriptions and results
Appropriateness of referral was defined by the proportion of patients referred who were diagnosed with cancer. The approach to implement an eCDST within GP workflow varied, but the tools were designed to be used in real time, during consultation, or applied outside of the consultation to flag up potential cases of cancer.
Quantitative synthesis of the included studies was not possible because of significant methodological and clinical heterogeneity. Results for the risk of bias assessment for each included study are given in Table 2. In summary, four quantitative studies had a low risk of bias, including the quantitative component of the mixed-methods study,26–28,35 one had a moderate risk,25 and in two the risk of bias was high.24,29 Of the qualitative studies, three had a low risk of bias32–34 and one a high risk, including the qualitative component of the mixed-methods paper.35 High risk of bias did not influence the inclusion of articles in the review.
Quantitative
eCDSTs used during GP consultation
Three studies26,28,35 examined eCDSTs that were designed to be used during the GP consultation with patients. Jiwa et al35 assessed whether an electronic referral pro forma used when patients present with bowel symptoms improved the appropriateness of referral for colorectal cancer (CRC). Control practices received an educational outreach visit by a local colorectal surgeon. The pro forma had no impact on the appropriateness of referral; however, it did improve the information and quality of the referral in comparison with the standard referral used by the control group.
Logan et al26 used a computer-generated prompt that recommended further investigations to rule out CRC when full blood-count results indicated iron deficiency anaemia. Control practices received laboratory results as per ‘usual care’. The prompts had no effect on the appropriateness of referral or investigation for CRC but, instead, led to increased prescriptions and adequate dose of iron.
Walter et al28 assessed MoleMate, a diagnostic tool for melanoma, which incorporates a scoring algorithm with spectrophotometric intracutaneous analysis (also known as SIAscopy) of pigmented skin lesions. MoleMate did not improve the appropriateness of referral, due in part to the high sensitivity and relatively low specificity set for the eCDST.28 Despite this, a health economic analysis found that, in UK practice, MoleMate was likely to be cost-effective compared with current best practice with an incremental cost-effectiveness ratio of £1896 per quality-adjusted life year gained.31
eCDSTs used outside the GP consultation
Two studies assessed eCDSTs designed to be applied outside of the consultation, identifying patients at increased risk of an undiagnosed cancer. Murphy et al27 applied electronic triggers to identify ‘red-flag’ symptoms in patients who had presented to their GP in the previous 90 days, without documented follow-up. There was a statistically significant reduction in time to diagnostic evaluation for CRC and prostate cancer in the intervention arm.
Kidney et al25 evaluated an eCDST that searched the patient’s electronic medical record and created a list of patients at increased risk of an undiagnosed CRC based on National Institute for Health and Care Excellence guidelines for urgent referrals. A third of all patients flagged by the algorithm were judged to need further review by their GP; 1.2% were subsequently diagnosed with CRC.
Clinical images
Two studies24,29 — both of low quality — that tested eCDSTs to support GPs’ assessment of clinical images of melanoma and non-melanoma skin cancer were identified. Both studies showed an improvement in decision making for cancerous and non-cancerous lesions.
Qualitative
Three studies showed improvements in decision making related to cancer diagnosis,24,25,29 one showed reduced time to diagnosis,27 and three demonstrated positive effects on secondary clinical or health service outcomes such as prescribing,26 quality of referrals,35 or cost-effectiveness.31 Jiwa et al35 and Kidney et al34 conducted a qualitative sub-study within their quantitative evaluation of the eCDST, both involving semi-structured interviews with GPs. Chiang et al33 and Dikomitis et al32 conducted exploratory qualitative studies of GPs’ experiences using eCDSTs in practice, one of which used simulated consultations. Both studies used NPT as a framework. The themes and constructs extracted from each qualitative study are outlined in Table 3.
Table 3. Qualitative results: normalisation process theory
Overarching themes
Three core constructs were identified in the synthesis:
Mistrust of the eCDST was driven by the disagreement between the tool’s recommendations and the GP’s assessment, ambiguity of underlying guidelines embedded within the eCDST, and a desire to understand the evidence that underpinned the clinical recommendation.32,33,35
The GP’s role as a gatekeeper was identified as a barrier due to conflicting referral thresholds between the eCDST and the GP, with GPs concerned about potential over-referral of patients at low risk.32,34,35 Finally, for eCDSTs designed to be used during consultation, there were challenges due to disruption of the usual workflow and the generation of additional tasks in an already-busy appointment.32–34
DISCUSSION
Summary
This systematic review evaluated the efficacy of eCDSTs used for cancer diagnosis in primary care, and describes factors that influence effective implementation. Three studies showed improvements in decision making related to cancer diagnosis,24,25,29 one showed reduced time to diagnosis,27 and three demonstrated positive effects on secondary clinical or health service outcomes such as prescribing26, quality of referrals,35 or cost-effectiveness.31 Key qualitative findings related to issues of trust in the tool, the impact on a GP’s role as gatekeeper, and potential negative effects on GP workflow.
eCDSTs that were used outside of GP consultations appeared to be more effective than tools used in real-time during consultation; they seemed to have the ability to detect patients at an increased risk of an undiagnosed cancer, leading to improvements in clinical assessment and time to diagnostic assessment. However, the implementation issues shifted from a disruption of the GP workflow during consultation to the ability to successfully convey the results of the eCDST to GPs outside of the consultation and ensure they acted on the information.25,27 Communicating this information to GPs did not always lead to follow-up of the patient.
Strengths and limitations
To the authors’ knowledge, this is the first review to evaluate the efficacy of eCDSTs used for cancer diagnosis in primary care and examine factors influencing their effective implementation. Rigorously conducted, it provides a summary of available findings to inform eCDST implementation in primary care for the diagnosis of cancer.
However, there are some limitations. This review is limited by the small number of included studies and large-scale randomised controlled trials: eCDSTs are relatively under-utilised for cancer diagnosis. Further, none of the included studies looked at outcomes such as survival rates and only one evaluated time to diagnosis;27 this highlights the challenges of conducting trials of diagnostic interventions for relatively rare conditions in primary care. Much larger implementation trials with long-term follow-up of cancer diagnoses, stage, and survival are required to determine the magnitude or effect of eCDSTs on cancer outcomes.
Comparison with existing literature
As with this work, a 2011 systematic review by Mansell et al36 did not identify any studies that examined a delay in referral of cancer as a primary outcome; all 22 included studies used a proxy measurement, such as GP knowledge or quality of referrals.
Mistrust of the eCDST was driven by the disagreement between the tool’s suggestion and the GP’s assessment, ambiguity of guidelines, and a desire to understand the underlying research underpinning the clinical recommendation.32,33 GPs reported that the eCDST compromised their autonomy, with the eCDST recommendation being perceived as ‘the final word’ rather than support at the time of decision making.33 This is consistent with recent evidence from the GUIDES implementation guidelines for eCDSTs.37 These guidelines comprise a checklist of factors that were found by patient and healthcare users to influence the effectiveness of eCDST implementation. The GUIDES checklist highlighted that the most important factor for successful implementation is ‘trustworthy evidence-based information’.37
As gatekeepers, there is much pressure on GPs to balance the use of limited and costly referrals for tests against potentially missing a cancer diagnosis.38 There are conflicting thresholds when comparing an eCDST’s output with the GP’s ability to refer everyone who was recommended.32–34 The International Cancer Benchmarking Partnership, a collaboration between six countries, identified that a stronger gatekeeper role, and different cancer-risk thresholds for referral, were associated with poorer cancer survival.39 Concerns about resource constraints and unwillingness to refer differed by country, but was found to play a large role in decision making in Australia and the UK.39
The usability and acceptability of eCDSTs was dependent on several competing issues, such as disruption of workflow, prompt fatigue, and time. There is a growing recognition in the literature that the technology being developed must seamlessly integrate into the current work practices of those using eCDSTs.37 The eCDST’s functionality and how it affects workflow could be mitigated using a consistent feedback loop between GPs and tool designers.40 There were no practices in place to monitor and adapt the eCDSTs for use in consultation, and no opportunity for the GPs to critically appraise how the tool affects workflow.
Implications for research and practice
The diagnostic algorithms in the eCDSTs included in this review were of a limited nature, but diagnostic and clinical utility could increase with more sophisticated algorithms that combine a larger number of factors such as symptoms, abnormal test results, and patterns over time. With the advances in artificial intelligence (AI) and machine learning in clinical practice, future developments will likely drive the next wave of eCDSTs. The use of AI has had promising preliminary results in areas such as visual image analysis in dermatology41 and radiology;42 however, further research using large primary care datasets is required before it can be known whether this approach will have utility to improve diagnoses of symptomatic cancers in general practice.
The available evidence in this review suggests that eCDSTs have the capacity to improve decision making for a cancer diagnosis, but the optimal mode of delivery remains unclear. Given the complex nature of a cancer diagnosis, the advancement and sustainability of eCDSTs in primary care relies on a continuous loop of practitioner feedback and refinement. The findings of the review presented here indicate that improvements in their design and implementation are needed to ensure they can be embedded in normal general practice workflows and alter professional decision making as intended. Strategies for effective communication need to be better explored.
Notes
Funding
This work was supported by the Cancer Australia Primary Care Cancer Collaborative Clinical Trials Group (PC4). Jon D Emery is supported by a National Health and Medical Research Council Practitioner Fellowship and is a member of the senior faculty of the multi-institutional CanTest Collaborative, which is funded by Cancer Research UK (C8640/A23385).
Ethical approval
Ethical approval was not required for this study.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors have declared no competing interests.
- Received June 18, 2019.
- Revision requested July 29, 2019.
- Accepted August 26, 2019.
- © British Journal of General Practice 2019