Elsevier

Social Science & Medicine

Volume 83, April 2013, Pages 27-33
Social Science & Medicine

Visual representation of statistical information improves diagnostic inferences in doctors and their patients

https://doi.org/10.1016/j.socscimed.2013.01.034Get rights and content

Abstract

Doctors and patients have difficulty inferring the predictive value of a medical test from information about the prevalence of a disease and the sensitivity and false-positive rate of the test. Previous research has established that communicating such information in a format the human mind is adapted to—namely natural frequencies—as compared to probabilities, boosts accuracy of diagnostic inferences. In a study, we investigated to what extent these inferences can be improved—beyond the effect of natural frequencies—by providing visual aids. Participants were 81 doctors and 81 patients who made diagnostic inferences about three medical tests on the basis of information about prevalence of a disease, and the sensitivity and false-positive rate of the tests. Half of the participants received the information in natural frequencies, while the other half received the information in probabilities. Half of the participants only received numerical information, while the other half additionally received a visual aid representing the numerical information. In addition, participants completed a numeracy scale. Our study showed three important findings: (1) doctors and patients made more accurate inferences when information was communicated in natural frequencies as compared to probabilities; (2) visual aids boosted accuracy even when the information was provided in natural frequencies; and (3) doctors were more accurate in their diagnostic inferences than patients, though differences in accuracy disappeared when differences in numerical skills were controlled for. Our findings have important implications for medical practice as they suggest suitable ways to communicate quantitative medical data.

Highlights

► Doctors and patients make more accurate inferences when risks are communicated in natural frequencies than probabilities. ► Visual aids improve accuracy even when risks are provided in natural frequencies. ► Differences in accuracy between doctors and patients disappear when differences in levels of numeracy are controlled for. ► Our findings suggest suitable ways to communicate quantitative medical data.

Introduction

Doctors and patients have difficulty inferring the positive predictive value of a medical test from information about the prevalence of a disease and the sensitivity and false-positive rate of the test. To illustrate, in an influential study on how doctors process information about the results of mammography, Eddy (1982) gave 100 doctors the following information: “The probability that a woman has breast cancer is 1%. When a woman has breast cancer, it is not sure that she will have a positive result on the mammography: she has an 80% probability of having a positive result on the mammography. When a woman does not have breast cancer, it is still possible that she will have a positive result on the mammography: she has a 10% probability of having a positive result on the mammography.” After having seen this information, doctors were required to estimate the probability that a woman with a positive mammography actually has breast cancer. Eddy reported that 95 of 100 doctors estimated this probability to be about 80%. If one inserts the numbers presented above into Bayes' theorem, however, one gets a value of 8%, that is, an estimate one order of magnitude smaller (Eddy, 1982).

Previous research has established that communicating risk information in a format the human mind is adapted to—namely natural frequencies instead of probabilities—improves inferences considerably (Gigerenzer & Hoffrage, 1995). Natural frequencies are final tallies in a set of objects or events randomly sampled from the natural environment (Hoffrage, Gigerenzer, Krauss, & Martignon, 2000). For the mammography task and for a (fictitious) sample of 10,000 women, the statistical information provided in terms of natural frequencies reads: “One hundred out of every 10,000 women have breast cancer. When a woman has breast cancer, it is not sure that she will have a positive result on the mammography: 80 of every 100 such women will have a positive result on the mammography. When a woman does not have breast cancer, it is still possible that she will have a positive result on the mammography: 990 out of every 9900 such women will have a positive result on the mammography.”

Gigerenzer and Hoffrage (1995) investigated whether information presented in terms of natural frequencies helps students unfamiliar with Bayes' theorem to find the Bayesian response. In 15 different inferential tasks, including the mammography problem, the percentage of Bayesian responses increased from about 10–20% to about 50%. Since then, this effect has been replicated many times (see Hoffrage & Gigerenzer, 1998, for medical doctors, and Hoffrage, Lindsey, Hertwig, & Gigerenzer, 2000, for medical students, lawyers, and law students; see also Brase, 2008).

Even though the effect of numerical format (probabilities vs. natural frequencies) is substantial, the performance in the natural frequency condition still leaves room for improvement. Another method that has been proposed as potentially promising for improving the communication and understanding of risks is using visual aids, such as icon arrays, grids, and bar graphs to display information graphically (Ancker, Senathirajah, Kukafka, & Starren, 2006; Lipkus, 2007; Paling, 2003; see Garcia-Retamero & Galesic, 2013, for a review). Visual aids can improve comprehension of risks associated with different medical treatments, screenings, and lifestyles, promoting consideration of beneficial treatments despite side-effects (Elting, Martin, Cantor, & Rubenstein, 1999; Waters, Weinstein, Colditz, & Emmons, 2007; Zikmund-Fisher, Fagerlin, & Ubel, 2008). Visual aids can also increase beneficial risk-avoidance (Schirillo & Stone, 2005), promote healthy behaviors (Cox, Cox, Sturm, & Zimet, 2010; Garcia-Retamero & Cokely, 2011), aid comprehension of complex concepts such as incremental risk (Zikmund-Fisher, Ubel, et al., 2008), and reduce errors induced by anecdotal narratives (Fagerlin, Wang, & Ubel, 2005) and biases (Garcia-Retamero & Dhami, 2011; in press; Garcia-Retamero & Galesic, 2009; Garcia-Retamero, Galesic, & Gigerenzer, 2010). Moreover, risk information presented via visual aids is perceived to be easier to understand and recall, and requires less viewing time than the same information presented numerically (Feldman-Stewart, Brundage, & Zotov, 2007; Gaissmaier et al., 2012; Goodyear-Smith et al., 2008). However, some caution is warranted as visual aids can also be misused to represent risk information in a misleading way (Ancker, Weber, & Kukafka, 2011; Kurz-Milcke, Gigerenzer, & Martignon, 2008; Stone et al., 2003). The first aim of the present paper was to investigate to what extent providing visual aids boosts the accuracy of diagnostic inferences in experienced medical doctors and their patients above and beyond the effect of natural frequencies.

A second aim of this paper was to explore the impact of numerical skills on the accuracy of diagnostic inferences, and whether these skills interact with the effect of natural frequencies. Numeracy refers to people's ability to understand and to deal with numerical information (Lipkus & Peters, 2009; Peters, 2012; Peters, Hibbard, Slovic, & Dieckmann, 2007; Reyna, Nelson, Han, & Dieckmann, 2009), and it has been measured with various scales (e.g., Cokely, Galesic, Schulz, Ghazal, & Garcia-Retamero, 2012; Lipkus, Samsa, & Rimer, 2001; Schwartz, Woloshin, Black, & Welch, 1997; Zikmund-Fisher, Smith, Ubel, & Fagerlin, 2007). The evidence regarding the question of how numeracy interacts with numerical format (frequencies vs. probabilities) is inconclusive. Using a vignette describing a psychiatric patient, Mr. Jones, and asking participants to assess the risk of releasing this patient, Peters et al. (2006) found that highly numerate participants were relatively unaffected by how numerical information was displayed (frequencies vs. probabilities), whereas less numerate participants judged the risk to be greater when information was presented in frequencies. Peters et al. (2006) interpreted these results by suggesting that highly numerate participants can more easily translate probability information into frequencies and vice versa, so that it does not matter that much in which format the information has initially been given to them. Schapira, Davids, McAuliffe, and Nattinger (2004) obtained a similar finding: While less numerate participants showed a relatively high degree of inconsistency between their breast cancer risk assessments based on probability vs. frequency information, the assessments of the highly numerate participants were relatively unaffected by the numerical format. This can easily be explained by participants' ability to convert numbers from one format to another. In contrast to these findings and to this interpretation, Chapman and Liu (2009) found that the beneficial effect of displaying information in terms of frequencies (instead of probabilities) was more pronounced for people with high numeracy scores than for those with low numeracy—an observation that Hill and Brase (2012) explained by what they called the threshold hypotheses: “Numeracy of the person must be above threshold for frequencies to help” (see Hill & Brase, 2012, Table 1).

The question of how numeracy relates to visual aids calls for more research as well. Investigating this issue was the third aim of this paper. Previous studies on the usefulness of visual aids have often focused on people with low levels of numeracy, cognitive capacity, or experience, that is, people who are more likely to be susceptible to biases in judgment and decision making (Bickmore, Pfeifer, & Paasche-Orlow, 2009; Reyna et al., 2009; Schwartz et al., 1997). For instance, in a study with undergraduates with no formal medical training, Brase (2009) showed that different types of visual aids improve diagnostic reasoning in a fictitious task. Similarly, in a survey of probabilistic, national samples in two different countries (United States and Germany), Garcia-Retamero and Galesic (2010) compared the effectiveness of adding different types of visual aids to numerical information about treatment risk reduction. Results showed large improvements in accuracy of risk understanding, regardless of which type of visual aids was used. Numeracy, however, also played an important role: visual aids were most useful for the participants with low levels of numeracy (see also Garcia-Retamero & Galesic, 2013; Hawley et al., 2008; Peters et al., 2009). It is unclear, however, whether visual aids would be as effective with the type of educated and experienced professional decision makers that we study here: medical doctors. It is also unclear whether the beneficial effects of visual aids on diagnostic inferences in doctors and their patients interacts with their numerical skills, that is, whether participants with high and low levels of numeracy profit to the same extent from visual aids.

Finally, we aimed to investigate whether visual aids not only affect objective inferences (i.e., accuracy of probability or frequency judgments) but also subjective perceptions (i.e., estimates of usefulness of information and task difficulty). A comparison between actual performance and subjective perceptions is reminiscent of studies on overconfidence, in which accuracy is compared to confidence of being correct (Gigerenzer, Hoffrage, & Kleinbölting, 1991).

Section snippets

Participants

Participants were 81 doctors and 81 patients. Doctors were recruited from four hospitals in the cities of Jaén and Granada (Spain). Patients were recruited by these doctors during treatment consultation. To be eligible for recruitment, doctors had to be general practitioners for at least 1 year, and patients had to have no previous formal medical training. If doctors and patients agreed to participate, they were provided with an introductory letter describing the purpose of the study, and their

Results

Can diagnostic inferences be improved in doctors and their patients beyond the effect of natural frequencies by providing visual aids? As Fig. 2 shows, performance was better when the information was provided in natural frequencies (overall 51% correct inferences, averaged across all tasks and all other conditions) as compared to probabilities (38%). Performance was also better when the information was presented both numerically and visually (62% correct inferences) as compared to numerically

Discussion

Our study showed four important findings: First, doctors and their patients made more accurate inferences from medical tests when they were provided with information in a format the human mind is adapted to—namely, natural frequencies—as compared to probabilities. Second, doctors and their patients made more accurate inferences when they received visual aids representing the numerical information as compared to receiving the same information only in numbers. In fact, the use of visual aids was

Acknowledgments

This research is part of the projects “How to Improve Understanding of Risks about Health (PSI2008-02019)” funded by the Ministerio de Ciencia e Innovación (Spain), and “Helping Doctors and Their Patients Make Decisions about Health (PSI2011-22954)” funded by the Ministerio de Economía y Competitividad (Spain).

References (62)

  • B.J. Zikmund-Fisher et al.

    Communicating side effect risks in a tamoxifen prophylaxis decision aid: the debiasing influence of pictographs

    Patient Education & Counseling

    (2008)
  • J.S. Ancker et al.

    Effect of arrangement of stick figures on estimates of proportion in risk graphics

    Medical Decision Making

    (2011)
  • A.J. Apter et al.

    Asthma numeracy skill and health literacy

    Journal of Asthma

    (2006)
  • A.J. Apter et al.

    Numeracy and communication with patients: they are counting on us

    Journal of General Internal Medicine

    (2008)
  • G.L. Brase

    Frequency interpretation of ambiguous statistical information facilitates Bayesian reasoning

    Psychonomic Bulletin & Review

    (2008)
  • G.L. Brase

    Pictorial representations in statistical reasoning

    Applied Cognitive Psychology

    (2009)
  • B.C. Chapman et al.

    Numeracy, frequency, and Bayesian reasoning

    Judgment & Decision Making

    (2009)
  • E.T. Cokely et al.

    Measuring risk literacy: the Berlin numeracy test

    Judgment & Decision Making

    (2012)
  • J.A. Covey

    Meta-analysis of the effects of presenting treatment benefits in different formats

    Medical Decision Making

    (2007)
  • D.S. Cox et al.

    Behavioral interventions to increase HPV vaccination acceptability among mothers of young girls

    Health Psychology

    (2010)
  • S.L. Davids et al.

    Predictors of pessimistic breast cancer risk perceptions in a primary care population

    Journal of General Internal Medicine

    (2004)
  • L. Donelle et al.

    Health literacy and numeracy: key factors in cancer risk comprehension

    Chronic Diseases in Canada

    (2008)
  • D.M. Eddy

    Probabilistic reasoning in clinical medicine: problems and opportunities

  • L. Elting et al.

    Influence of data display formats on physician investigators' decisions to stop clinical trials: prospective trial with repeated measures

    British Medical Journal

    (1999)
  • A. Fagerlin et al.

    Making numbers matter: present and future research in risk communication

    American Journal of Health Behavior

    (2007)
  • A. Fagerlin et al.

    Reducing the influence of anecdotal reasoning on people's health care decisions: is a picture worth a thousand statistics?

    Medical Decision Making

    (2005)
  • D. Feldman-Stewart et al.

    Further insight into the perception of quantitative information: judgments of gist in treatment decisions

    Medical Decision Making

    (2007)
  • W. Gaissmaier et al.

    Numbers can be worth a thousand pictures: individual differences in understanding graphical and numerical representations of health-related information

    Health Psychology

    (2012)
  • M. Galesic et al.

    Statistical numeracy for health: a cross-cultural comparison with probabilistic national samples

    Archives of Internal Medicine

    (2010)
  • M. Galesic et al.

    Do low-numeracy people avoid shared decision making?

    Health Psychology

    (2011)
  • R. Garcia-Retamero et al.

    Effective communication of risks to young adults: using message framing and visual aids to increase condom use and STD screening

    Journal of Experimental Psychology: Applied

    (2011)
  • Cited by (130)

    • Decision aids for patients with carotid stenosis

      2024, Journal of Vascular Surgery
    View all citing articles on Scopus
    View full text