Abstract
OBJECTIVE
To synthesise current evidence for the influence on clinical behaviour of patient-specific electronically generated reminders available at the time of the clinical encounter.
DATA SOURCES
PubMed, Cochrane library of systematic reviews; Science Citation Index Expanded; Social Sciences Citation Index; ASSIA; EMBASE; CINAHL; DARE; HMIC were searched for relevant articles.
STUDY ELIGIBILITY CRITERIA, PARTICIPANTS AND INTERVENTIONS
We included controlled trials of reminder interventions if the intervention was: directed at clinician behaviour; available during the clinical encounter; computer generated (including computer generated paper-based reminders); and generated by patient-specific (rather than condition specific or drug specific) data.
STUDY APPRAISAL AND SYNTHESIS METHODS
Systematic review and meta-analysis of controlled trials published since 1970. A random effects model was used to derive a pooled odds ratio for adherence to recommended care or achievement of target outcome. Subgroups were examined based on area of care and study design. Odds ratios were derived for each sub-group. We examined the designs, settings and other features of reminders looking for factors associated with a consistent effect.
RESULTS
Altogether, 42 papers met the inclusion criteria. The studies were of variable quality and some were affected by unit of analysis errors due to a failure to account for clustering. An overall odds ratio of 1.79 [95% confidence interval 1.56, 2.05] in favour of reminders was derived. Heterogeneity was high and factors predicting effect size were difficult to identify.
LIMITATIONS
Methodological diversity added to statistical heterogeneity as an obstacle to meta-analysis. The quality of included studies was variable and in some reports procedural details were lacking.
CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS
The analysis suggests a moderate effect of electronically generated, individually tailored reminders on clinician behaviour during the clinical encounter. Future research should concentrate on identifying the features of reminder interventions most likely to result in the target behaviour.
Similar content being viewed by others
References
Balas EA, Austin SM, Mitchell JA, Ewigman BG, Bopp KD, Brown GD. The clinical value of computerized information services - A review of 98 randomized clinical trials. Arch Fam Med. 1996;5(5):271–8.
Bennett JW, Glasziou PP, Sim I. Review: Computerised reminders and feedback can improve provider medication management. Evid Base Med. 2003;8(6):190.
Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. JAMA. 2005;293(10):1223–38.
Kawamoto K, Houlihan C, Balas E, Lobach D. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765.
Mitchell E, Sullivan F. A descriptive feast but an evaluative famine: systematic review of published articles on primary care computing during 1980-97. BMJ. 2001;322(7281):279–82E.
Montgomery AA, Fahey T. A systematic review of the use of computers in the management of hypertension. Journal of Epidemiology and Community Health. 1998;52(8):520–5.
Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc. 1996;3(6):399–409.
Shiffman RN, Liaw Y, Brandt CA, Corb GJ. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc. 1999;6(2):104–14.
Dexheimer JW, Talbot TR, Sanders DL, Rosenbloom ST, Aronsky D. Prompting clinicians about preventive care measures: a systematic review of randomized controlled trials. J Am Med Inform Assoc. 2008;15(3):311–20.
Shojania KG, Jennings A, Mayhew A, et al. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev. 2009;3:CD001096.
Shojania KG, Jennings A, Mayhew A, et al. Effect of point-of-care computer reminders on physician behaviour: a systematic review. CMAJ. 2010;182:E216–25.
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org (last accessed 24.1.12).
Adams G, Gulliford MC, Ukoumunne OC, et al. Patterns of intra-cluster correlation from primary care research to inform study design and analysis. J Clin Epidemiol. 2004;57(8):785–794.
Matheny ME, Sequist TD, Seger AC, Fiskio JM, Sperling M, Bugbee D, et al. A randomized trial of electronic clinical reminders to improve medication laboratory monitoring. J Am Med Inform Assoc. 2008;15(4):424–9.
Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma'luf N, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999;106(2):144–50.
Burack RC, Gimotty PA, George J, Simon MS, Dews P, Moncrease A. The effect of patient and physician reminders on use of screening mammography in a health maintenance organization. Results of a randomized controlled trial. Cancer. 1996;78(8):1708–21.
Burack RC, Gimotty PA, George J, McBride S, Moncrease A, Simon MS, et al. How reminders given to patients and physicians affected pap smear use in a health maintenance organization: results of a randomized controlled trial. Cancer. 1998;82(12):2391–400.
Chambers CV, Balaban DJ, Carlson BL, Grasberger DM. The effect of microcomputer-generated reminders on influenza vaccination rates in a university-based family practice center. J Am Board Fam Pract. 1991;4(1):19–26.
Dexter PR, Wolinsky FD, Gramelspacher GP, Zhou XH, Eckert GJ, Waisburd M, et al. Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms. A randomized, controlled trial. Ann Intern Med. 1998;128(2):102–10.
Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ. A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med. 2001;345(13):965–70.
Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, et al. Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002;325(7370):941.
Filippi A, Sabatini A, Badioli L, Samani F, Mazzaglia G, Catapano A, et al. Effects of an automated electronic reminder in changing the antiplatelet drug-prescribing behavior among Italian general practitioners in diabetic patients: an intervention trial. Diabetes Care. 2003;26(5):1497–500.
Frank O, Litt J, Beilby J. Opportunistic electronic reminders: improving performance of preventive care in general practice. Aust Fam Physician. 2004;33(1/2):87–90.
Hicks LS, Sequist TD, Ayanian JZ, Shaykevich S, Fairchild DG, Orav EJ, et al. Impact of computerized decision support on blood pressure management and control: A randomized controlled trial. J Gen Intern Med. 2007;23(4):429–41.
Judge J, Field TS, DeFlorio M, Laprino J, Auger J, Rochon P, et al. Prescribers' responses to alerts during medication ordering in the long term care setting. J Am Med Inform Assoc. 2006;13(4):385–90.
Kenealy T, Arroll B, Petrie KJ. Patients and computers as reminders to screen for diabetes in family practice. Randomized-controlled trial. J Gen Intern Med. 2005;20(10):916–21.
Kralj B, Iverson D, Hotz K, Ashbury FD. The impact of computerized clinical reminders on physician prescribing behavior: Evidence from community oncology practice. Am J Med Qual. 2003;18(5):197–203.
Krall MA, Traunweiser K, Towery W. Effectiveness of an electronic medical record clinical quality alert prepared by off-line data analysis. Medinfo. 2004;11(Pt 1):135–9.
Kucher N, Koo S, Quiroz R, Cooper JM, Paterno MD, Soukonnikov B, et al. Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med. 2005;352(10):969–77.
Litzelman DK, Dittus RS, Miller ME, Tierney WM. Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. J Gen Intern Med. 1993;8(6):311–7.
Lo HG, Matheny ME, Seger DL, Bates DW, Gandhi TK. Impact of non-interruptive medication laboratory monitoring alerts in ambulatory care. J Am Med Inform Assoc. 2009;16(1):66–71.
McCowan C, Neville RG, Ricketts IW, Warner FC, Hoskins G, Thomas GE. Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma. Med Inform Internet Med. 2001;26(3):191–201.
McDonald CJ. Use of a computer to detect and respond to clinical events: Its effect on clinical behavior. Ann Intern Med. 1976;84:162–7.
McDonald CJ, Wilson GA, McCabe GP Jr. Physician response to computer reminders. JAMA. 1980;244(14):1579–81.
McDowell I, Newell C, Rosser W. A randomized trial of computerized reminders for blood pressure screening in primary care. Med Care. 1989;27(3):297–305.
McDowell I, Newell C, Rosser W. Computerized reminders to encourage cervical screening in family practice. J Fam Pract. 1989;28(4):420–4.
Murray MD, Harris LE, Overhage JM, Zhou XH, Eckert GJ, Smith FE, et al. Failure of Computerized Treatment Suggestions to Improve Health Outcomes of Outpatients with Uncomplicated Hypertension: Results of a Randomized Controlled Trial. Pharmacotherapy. 2004;24(3):324–37.
Overhage JM, Tierney WM, McDonald CJ. Computer reminders to implement preventive care guidelines for hospitalized patients. Arch Intern Med. 1996;156(14):1551–6.
Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Inform Assoc. 1997;4(5):364–75.
Rosser WW, McDowell I, Newell C. Use of Reminders for Preventive Procedures in Family Medicine. CMAJ. 1991;145(7):807.
Rosser WW, Hutchison BG, McDowell I, Newell C. Use of Reminders to Increase Compliance with Tetanus Booster Vaccination. CMAJ. 1992;146(6):911–7.
Rossi RA, Every NR. A computerized intervention to decrease the use of calcium channel blockers in hypertension. J Gen Intern Med. 1997;12(11):672–8.
Rothschild J, McGurk S, Honour M, Lu L, McClendon A, Srivastava P, et al. Assessment of education and computerized decision support interventions for improving transfusion practice. Transfusion. 2007;47(2):228–39.
Safran C, Rind DM, Davis RB, Ives D, Sands DZ, Currier J, et al. Guidelines for management of HIV infection with computer-based patient's record. Lancet. 1995;346(8971):341–6.
Sequist TD, Gandhi TK, Karson AS, Fiskio JM, Bugbee D, Sperling M, et al. A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. Journal of the American Medical Informatics Association. 2005;12(4):431–7.
Shea S, Sideli RV, DuMouchel W, Pulver G, Arons RR, Clayton PD. Computer-generated informational messages directed to physicians: effect on length of hospital stay. J Am Med Inform Assoc. 1995;2(1):58–64.
Tamblyn R, Huang A, Perreault R, Jacques A, Roy D, Hanley J, et al. The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. CMAJ. 2003;169(6):549–56.
Tamblyn R, Huang A, Taylor L, Kawasumi Y, Bartlett G, Grad R, et al. A randomized trial of the effectiveness of on-demand versus computer-triggered drug decision support in primary care. J Am Med Inform Assoc. 2008;15(4):430–8.
Tape TG, Campbell JR. Computerized medical records and preventive health care: success depends on many factors. Am J Med. 1993;94(6):619–25.
Tierney WM, McDonald CJ, Martin DK, Rogers MP. Computerized display of past test results. Effect on outpatient testing. Ann Intern Med. 1987;107(4):569–74.
Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, et al. Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med. 2003;18(12):967–76.
Tierney W, Overhage J, Murray M, Harris L, Zhou X, Eckert G, et al. Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res. 2005;40(2):477–97.
van Wyk MA, van der Lei J, Mosseveld M, Bohnen AM, van Bemmel JH. Assessment of decision support for blood test ordering in primary care. A randomized trial. Ann Intern Med. 2001;134(4):274–81.
White KS, Lindsay A, Pryor TA, Brown WF, Walsh K. Application of a computerized medical decision-making process to the problem of digoxin intoxication. J Am Coll Cardiol. 1984;4(3):571–6.
Holt TA, Thorogood M, Griffiths F, Munday S, Friede T, Stables D. Automated electronic reminders to facilitate cardiovascular disease prevention: randomised controlled trial. Brit J Gen Pract. 2010;60(573):e137–43.
Gandhi TJ, Sequist TD, Poon EG, et al. Primary care clinicians’ attitudes towards electronic clinical reminders and clinical practice guidelines. Proceedings of the AMIA Symposium 2003:848.
Krall MA, Sittig DF. Subjective assessment of usefulness and appropriate presentation mode of alerts and reminders in the outpatient setting. Proceedings of the AMIA Symposium 2001:334–338.
Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SM. Exploring barriers and facilitators to the use of computerized clinical reminders. JAMIA. 2005;12:438–447.
Patterson ES, Doebbeling BN, Fung CH, Militello L, Anders S, Asch SM. Identifying barriers to the effective use of clinical reminders: bootstrapping multiple methods. J Biomed Inform. 2005;38:189–199.
Sittig DF, Krall MA, Dykstra RH, Russell A, Chin HL. A survey of factors affecting clinician acceptance of clinical decision support. BMC Med Informat Decis Making. 2006;6:6.
Krall MA, Sittig DF. Clinicians’ assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements. Proceedings of the AMIA Symposium 2002:400–404.
Agrawal A, Mayo-Smith MF. Adherence to computerized clinical reminders in a large healthcare delivery network. Medinfo. 2004;11(1):111–114.
Mayo-Smith MF, Agrawal A. Factors associated with improved completion of computerized clinical reminders across a large healthcare system. Int J Med Informat. 2007;76(10):710–716.
Acknowledgements
We thank Dr Simon Gates for analytical advice and Samantha Johnson for assistance with database searching and retrieval of papers. We also thank the authors of studies for which clarification was required for providing useful information.
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Funding source
Funded through internal sources. No external funding.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Holt, T.A., Thorogood, M. & Griffiths, F. Changing Clinical Practice Through Patient Specific Reminders Available at the Time of the Clinical Encounter: Systematic Review and Meta-Analysis. J GEN INTERN MED 27, 974–984 (2012). https://doi.org/10.1007/s11606-012-2025-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-012-2025-5