Statistical Illiteracy in Residents: What They Do Not Learn Today Will Hurt Their Patients Tomorrow

Health care providers are the most highly rated sources of health information.1 Thus, responsibility for helping patients understand the potential benefits and harms of medical prevention and intervention falls largely upon physicians. Can physicians fulfill this task? The news on that front is not encouraging. Eddy2 reported already three decades ago that 95 out of 100 physicians overestimated the probability of breast cancer after a positive mammography test result by an order of magnitude. More recently, Bramwell and colleagues3 found that only 1 out of 21 obstetricians were able to estimate the probability of an unborn child actually having Down syndrome given a positive test; those most confident in their estimates were furthest away from the correct response. Naylor et al4 showed that physicians rated the effectiveness of a treatment much higher when the benefit was described in terms of a relative risk reduction—“a medical intervention results in a 34% relative decrease in the incidence of fatal and nonfatal myocardial infarction”—than when it was described as an absolute risk reduction—“a medical intervention results in a 1.4% decrease in the incidence of fatal and nonfatal myocardial infarction—2.5% vs. 3.9%.” None of 20 German gynecologists were aware of the most serious risk of mammography overdiagnosis, when asked specifically about the harms of that screening program.5 In a survey of 50 Australian physicians, only 8 were able to understand and explain number needed to treat to others.6 And 70% of 412 US primary care physicians based their recommendation in favor of a screening test on 5-year survival rates, although this is a misleading metric in the context of screening.7 All of these studies document the same phenomenon: A considerable number of physicians are statistically illiterate, that is, they do not understand the statistics of their own discipline.

Given the evidence corroborating the statistical innumeracy of many fully licensed physicians, the results of the study by Anderson and colleagues8 on statistical illiteracy in residents reported in this issue of the Journal of Graduate Medical Education may come as no surprise. Only 12% of the 4713 surveyed obstetrics-gynecology residents were able to correctly answer 2 simple questions on medical statistics: one on the meaning of the P value and the other on assessing the positive predictive value of mammography screening. The practical relevance of the latter question is particularly striking. What will the uninformed 88% of these residents say when their first patient asks about her chance of truly having breast cancer given a positive mammogram?

Statistically illiterate physicians are doomed to rely on their statistically illiterate conclusions, on local custom, and on the (mostly) inaccurate promises of pharmaceutical sales representatives and their leaflets. The price for this innumeracy is paid by patients who undergo medical procedures without being correctly informed of what to expect. When Schwartz and colleagues9 investigated a stratified sample of 479 American women, of whom most underwent regular mammography screening, 92% believed that this screening could not harm a woman without breast cancer. In fact, for every woman saved from breast cancer death by mammography, 10 healthy women, who would not have been diagnosed without screening, will be wrongly diagnosed with breast cancer and treated unnecessarily for it.10 The results of a representative study of 9 European countries with more than 10 000 participants amplify the depressing evidence of the US study: Frequent consultation with a physician was not correlated with better understanding of screening benefit but rather with overestimating it.11 

What to do? The study of Anderson et al8 identifies the Achilles' heel of physicians' statistical illiteracy: their medical training. Nearly 75% of surveyed residents considered their statistical education as adequate. At the same time, nearly 90% of surveyed residents got the 2 simple statistical questions wrong. As far back as 1937, an editorial in The Lancet stressed the importance of statistics in medicine and criticized physicians' “educational blind spot.” Yet, more than 75 years later, there is still no ongoing initiative from the Association of American Medical Colleges to address statistical literacy in medical school education.12 Medicine holds a long-standing antagonism toward statistics and many in the profession dismiss medical statistics as impersonal or irrelevant to the individual patient. But they are not. Imagining oneself in the situation of the above-mentioned patient, with a positive mammogram, makes clear how personal and individually relevant medical statistics can be. How fortunate the patient who has a physician who is statistically literate. This physician would be able to tell her that her chance of having breast cancer after a positive mammogram is about 10%, which may allay her worry while awaiting the results of further testing. Her odds of meeting such a physician are regrettably low.

Ensuring future patients receive the counseling they need is not only the responsibility of medical educators, who undoubtedly need to improve the quality of teaching medical statistics, it is also a vital responsibility of every medical student and licensed physician to remain curious about statistical issues he or she has not yet understood and to demand and take seriously high-quality statistical education from medical schools, residency programs, and specialty organizations. With a change in physician expectations, odds are that patients will benefit greatly.