Visual representation of statistical information improves diagnostic inferences in doctors and their patients
Highlights
► Doctors and patients make more accurate inferences when risks are communicated in natural frequencies than probabilities. ► Visual aids improve accuracy even when risks are provided in natural frequencies. ► Differences in accuracy between doctors and patients disappear when differences in levels of numeracy are controlled for. ► Our findings suggest suitable ways to communicate quantitative medical data.
Introduction
Doctors and patients have difficulty inferring the positive predictive value of a medical test from information about the prevalence of a disease and the sensitivity and false-positive rate of the test. To illustrate, in an influential study on how doctors process information about the results of mammography, Eddy (1982) gave 100 doctors the following information: “The probability that a woman has breast cancer is 1%. When a woman has breast cancer, it is not sure that she will have a positive result on the mammography: she has an 80% probability of having a positive result on the mammography. When a woman does not have breast cancer, it is still possible that she will have a positive result on the mammography: she has a 10% probability of having a positive result on the mammography.” After having seen this information, doctors were required to estimate the probability that a woman with a positive mammography actually has breast cancer. Eddy reported that 95 of 100 doctors estimated this probability to be about 80%. If one inserts the numbers presented above into Bayes' theorem, however, one gets a value of 8%, that is, an estimate one order of magnitude smaller (Eddy, 1982).
Previous research has established that communicating risk information in a format the human mind is adapted to—namely natural frequencies instead of probabilities—improves inferences considerably (Gigerenzer & Hoffrage, 1995). Natural frequencies are final tallies in a set of objects or events randomly sampled from the natural environment (Hoffrage, Gigerenzer, Krauss, & Martignon, 2000). For the mammography task and for a (fictitious) sample of 10,000 women, the statistical information provided in terms of natural frequencies reads: “One hundred out of every 10,000 women have breast cancer. When a woman has breast cancer, it is not sure that she will have a positive result on the mammography: 80 of every 100 such women will have a positive result on the mammography. When a woman does not have breast cancer, it is still possible that she will have a positive result on the mammography: 990 out of every 9900 such women will have a positive result on the mammography.”
Gigerenzer and Hoffrage (1995) investigated whether information presented in terms of natural frequencies helps students unfamiliar with Bayes' theorem to find the Bayesian response. In 15 different inferential tasks, including the mammography problem, the percentage of Bayesian responses increased from about 10–20% to about 50%. Since then, this effect has been replicated many times (see Hoffrage & Gigerenzer, 1998, for medical doctors, and Hoffrage, Lindsey, Hertwig, & Gigerenzer, 2000, for medical students, lawyers, and law students; see also Brase, 2008).
Even though the effect of numerical format (probabilities vs. natural frequencies) is substantial, the performance in the natural frequency condition still leaves room for improvement. Another method that has been proposed as potentially promising for improving the communication and understanding of risks is using visual aids, such as icon arrays, grids, and bar graphs to display information graphically (Ancker, Senathirajah, Kukafka, & Starren, 2006; Lipkus, 2007; Paling, 2003; see Garcia-Retamero & Galesic, 2013, for a review). Visual aids can improve comprehension of risks associated with different medical treatments, screenings, and lifestyles, promoting consideration of beneficial treatments despite side-effects (Elting, Martin, Cantor, & Rubenstein, 1999; Waters, Weinstein, Colditz, & Emmons, 2007; Zikmund-Fisher, Fagerlin, & Ubel, 2008). Visual aids can also increase beneficial risk-avoidance (Schirillo & Stone, 2005), promote healthy behaviors (Cox, Cox, Sturm, & Zimet, 2010; Garcia-Retamero & Cokely, 2011), aid comprehension of complex concepts such as incremental risk (Zikmund-Fisher, Ubel, et al., 2008), and reduce errors induced by anecdotal narratives (Fagerlin, Wang, & Ubel, 2005) and biases (Garcia-Retamero & Dhami, 2011; in press; Garcia-Retamero & Galesic, 2009; Garcia-Retamero, Galesic, & Gigerenzer, 2010). Moreover, risk information presented via visual aids is perceived to be easier to understand and recall, and requires less viewing time than the same information presented numerically (Feldman-Stewart, Brundage, & Zotov, 2007; Gaissmaier et al., 2012; Goodyear-Smith et al., 2008). However, some caution is warranted as visual aids can also be misused to represent risk information in a misleading way (Ancker, Weber, & Kukafka, 2011; Kurz-Milcke, Gigerenzer, & Martignon, 2008; Stone et al., 2003). The first aim of the present paper was to investigate to what extent providing visual aids boosts the accuracy of diagnostic inferences in experienced medical doctors and their patients above and beyond the effect of natural frequencies.
A second aim of this paper was to explore the impact of numerical skills on the accuracy of diagnostic inferences, and whether these skills interact with the effect of natural frequencies. Numeracy refers to people's ability to understand and to deal with numerical information (Lipkus & Peters, 2009; Peters, 2012; Peters, Hibbard, Slovic, & Dieckmann, 2007; Reyna, Nelson, Han, & Dieckmann, 2009), and it has been measured with various scales (e.g., Cokely, Galesic, Schulz, Ghazal, & Garcia-Retamero, 2012; Lipkus, Samsa, & Rimer, 2001; Schwartz, Woloshin, Black, & Welch, 1997; Zikmund-Fisher, Smith, Ubel, & Fagerlin, 2007). The evidence regarding the question of how numeracy interacts with numerical format (frequencies vs. probabilities) is inconclusive. Using a vignette describing a psychiatric patient, Mr. Jones, and asking participants to assess the risk of releasing this patient, Peters et al. (2006) found that highly numerate participants were relatively unaffected by how numerical information was displayed (frequencies vs. probabilities), whereas less numerate participants judged the risk to be greater when information was presented in frequencies. Peters et al. (2006) interpreted these results by suggesting that highly numerate participants can more easily translate probability information into frequencies and vice versa, so that it does not matter that much in which format the information has initially been given to them. Schapira, Davids, McAuliffe, and Nattinger (2004) obtained a similar finding: While less numerate participants showed a relatively high degree of inconsistency between their breast cancer risk assessments based on probability vs. frequency information, the assessments of the highly numerate participants were relatively unaffected by the numerical format. This can easily be explained by participants' ability to convert numbers from one format to another. In contrast to these findings and to this interpretation, Chapman and Liu (2009) found that the beneficial effect of displaying information in terms of frequencies (instead of probabilities) was more pronounced for people with high numeracy scores than for those with low numeracy—an observation that Hill and Brase (2012) explained by what they called the threshold hypotheses: “Numeracy of the person must be above threshold for frequencies to help” (see Hill & Brase, 2012, Table 1).
The question of how numeracy relates to visual aids calls for more research as well. Investigating this issue was the third aim of this paper. Previous studies on the usefulness of visual aids have often focused on people with low levels of numeracy, cognitive capacity, or experience, that is, people who are more likely to be susceptible to biases in judgment and decision making (Bickmore, Pfeifer, & Paasche-Orlow, 2009; Reyna et al., 2009; Schwartz et al., 1997). For instance, in a study with undergraduates with no formal medical training, Brase (2009) showed that different types of visual aids improve diagnostic reasoning in a fictitious task. Similarly, in a survey of probabilistic, national samples in two different countries (United States and Germany), Garcia-Retamero and Galesic (2010) compared the effectiveness of adding different types of visual aids to numerical information about treatment risk reduction. Results showed large improvements in accuracy of risk understanding, regardless of which type of visual aids was used. Numeracy, however, also played an important role: visual aids were most useful for the participants with low levels of numeracy (see also Garcia-Retamero & Galesic, 2013; Hawley et al., 2008; Peters et al., 2009). It is unclear, however, whether visual aids would be as effective with the type of educated and experienced professional decision makers that we study here: medical doctors. It is also unclear whether the beneficial effects of visual aids on diagnostic inferences in doctors and their patients interacts with their numerical skills, that is, whether participants with high and low levels of numeracy profit to the same extent from visual aids.
Finally, we aimed to investigate whether visual aids not only affect objective inferences (i.e., accuracy of probability or frequency judgments) but also subjective perceptions (i.e., estimates of usefulness of information and task difficulty). A comparison between actual performance and subjective perceptions is reminiscent of studies on overconfidence, in which accuracy is compared to confidence of being correct (Gigerenzer, Hoffrage, & Kleinbölting, 1991).
Section snippets
Participants
Participants were 81 doctors and 81 patients. Doctors were recruited from four hospitals in the cities of Jaén and Granada (Spain). Patients were recruited by these doctors during treatment consultation. To be eligible for recruitment, doctors had to be general practitioners for at least 1 year, and patients had to have no previous formal medical training. If doctors and patients agreed to participate, they were provided with an introductory letter describing the purpose of the study, and their
Results
Can diagnostic inferences be improved in doctors and their patients beyond the effect of natural frequencies by providing visual aids? As Fig. 2 shows, performance was better when the information was provided in natural frequencies (overall 51% correct inferences, averaged across all tasks and all other conditions) as compared to probabilities (38%). Performance was also better when the information was presented both numerically and visually (62% correct inferences) as compared to numerically
Discussion
Our study showed four important findings: First, doctors and their patients made more accurate inferences from medical tests when they were provided with information in a format the human mind is adapted to—namely, natural frequencies—as compared to probabilities. Second, doctors and their patients made more accurate inferences when they received visual aids representing the numerical information as compared to receiving the same information only in numbers. In fact, the use of visual aids was
Acknowledgments
This research is part of the projects “How to Improve Understanding of Risks about Health (PSI2008-02019)” funded by the Ministerio de Ciencia e Innovación (Spain), and “Helping Doctors and Their Patients Make Decisions about Health (PSI2011-22954)” funded by the Ministerio de Economía y Competitividad (Spain).
References (62)
- et al.
Design features of graphs in health risk communication: a systematic review
Journal of the American Medical Informatics Association
(2006) - et al.
Using computer agents to explain medical documents to patients with low health literacy
Patient Education & Counseling
(2009) - et al.
Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty
Cognition
(1996) - et al.
Literacy and numeracy skills and anticoagulation control
The American Journal of the Medical Sciences
(2004) - et al.
Communicating consequences of risky behaviors: life expectancy versus risk of disease
Patient Education & Counseling
(2011) - et al.
Who profits from visual aids: overcoming challenges in people's understanding of risks
Social Science & Medicine
(2010) - et al.
The impact of the format of graphical presentation on health-related knowledge and treatment choices
Patient Education & Counseling
(2008) - et al.
Numeracy, ratio bias, and denominator neglect in judgments of risk and probability
Learning & Individual Differences
(2008) - et al.
Patient understanding of food labels: the role of literacy and numeracy
American Journal of Preventive Medicine
(2006) - et al.
Foreground: background salience: explaining the effects of graphical displays on risk avoidance
Organizational Behavior & Human Decision Processes
(2003)
Communicating side effect risks in a tamoxifen prophylaxis decision aid: the debiasing influence of pictographs
Patient Education & Counseling
Effect of arrangement of stick figures on estimates of proportion in risk graphics
Medical Decision Making
Asthma numeracy skill and health literacy
Journal of Asthma
Numeracy and communication with patients: they are counting on us
Journal of General Internal Medicine
Frequency interpretation of ambiguous statistical information facilitates Bayesian reasoning
Psychonomic Bulletin & Review
Pictorial representations in statistical reasoning
Applied Cognitive Psychology
Numeracy, frequency, and Bayesian reasoning
Judgment & Decision Making
Measuring risk literacy: the Berlin numeracy test
Judgment & Decision Making
Meta-analysis of the effects of presenting treatment benefits in different formats
Medical Decision Making
Behavioral interventions to increase HPV vaccination acceptability among mothers of young girls
Health Psychology
Predictors of pessimistic breast cancer risk perceptions in a primary care population
Journal of General Internal Medicine
Health literacy and numeracy: key factors in cancer risk comprehension
Chronic Diseases in Canada
Probabilistic reasoning in clinical medicine: problems and opportunities
Influence of data display formats on physician investigators' decisions to stop clinical trials: prospective trial with repeated measures
British Medical Journal
Making numbers matter: present and future research in risk communication
American Journal of Health Behavior
Reducing the influence of anecdotal reasoning on people's health care decisions: is a picture worth a thousand statistics?
Medical Decision Making
Further insight into the perception of quantitative information: judgments of gist in treatment decisions
Medical Decision Making
Numbers can be worth a thousand pictures: individual differences in understanding graphical and numerical representations of health-related information
Health Psychology
Statistical numeracy for health: a cross-cultural comparison with probabilistic national samples
Archives of Internal Medicine
Do low-numeracy people avoid shared decision making?
Health Psychology
Effective communication of risks to young adults: using message framing and visual aids to increase condom use and STD screening
Journal of Experimental Psychology: Applied
Cited by (130)
Numerical versus graphical aids for decision-making in a multi-cue signal identification task
2024, Applied ErgonomicsDecision aids for patients with carotid stenosis
2024, Journal of Vascular SurgeryOn the generality and cognitive basis of base-rate neglect
2022, CognitionHypothetical case replacement can be used to quantify the robustness of trial results
2021, Journal of Clinical Epidemiology