Context.—Statistical literacy can be defined as understanding the statistical tests and terminology needed for the design, analysis, and conclusions of original research or laboratory testing. Little is known about the statistical literacy of clinical or anatomic pathologists.
Objective.—To determine the statistical methods most commonly used in pathology studies from the literature and to assess familiarity and knowledge level of these statistical tests by pathology residents and practicing pathologists.
Design.—The most frequently used statistical methods were determined by a review of 1100 research articles published in 11 pathology journals during 2015. Familiarity with statistical methods was determined by a survey of pathology trainees and practicing pathologists at 9 academic institutions in which pathologists were asked to rate their knowledge of the methods identified by the focused review of the literature.
Results.—We identified 18 statistical tests that appear frequently in published pathology studies. On average, pathologists reported a knowledge level between “no knowledge” and “basic knowledge” of most statistical tests. Knowledge of tests was higher for more frequently used tests. Greater statistical knowledge was associated with a focus on clinical pathology versus anatomic pathology, having had a statistics course, having an advanced degree other than an MD degree, and publishing research. Statistical knowledge was not associated with length of pathology practice.
Conclusions.—An audit of pathology literature reveals that knowledge of about 12 statistical tests would be sufficient to provide statistical literacy for pathologists. On average, most pathologists report they can interpret commonly used tests but are unable to perform them. Most pathologists indicated that they would benefit from additional statistical training.
The practice of both clinical and anatomic pathology requires familiarity with statistical concepts. The use of sophisticated statistical methods is vital for the modern academic pathologist from nearly all pathology disciplines, ranging from the critical appraisal of evidence and interpretation of results as dictated by evidence-based medicine (EBM), research (eg, genomics, biomarker studies, translational research, and health services research), and for laboratory operations, quality assurance, and improvement initiatives. All pathologists are also now required to complete a quality improvement project, which requires statistical knowledge for maintenance of certification. Additionally, reviewing manuscripts for journals requires some basic level of statistical familiarity to ensure that the data analysis methods presented in manuscripts are appropriate to address a research question and support the inferences made by authors.
Statistical literacy can be defined as understanding the statistical tests and terminology needed for the design, analysis, and conclusions of original research or laboratory testing.1 Several studies have found that physicians from a variety of medical disciplines have a poor understanding of statistics,2–7 even while believing such knowledge is important for their work.2,8 Evidence-based medicine training has been integrated into all levels of medical education and is mandated by the Accreditation Council for Graduate Medical Education (ACGME) for residents, but the ACGME has not provided specific guidelines regarding the content or methods for EBM training,9 and no published EBM curricula have been directed specifically to pathology. Although there are general concepts that underlie all statistics, different disciplines may emphasize particular techniques, and the use of statistical methods changes over time.10 Knowledge of how frequently statistical tests are used in published studies from a particular medical discipline could be useful in designing curricula and assessment tools specifically tailored for EBM in pathology.
Several studies have investigated how frequently different statistical methods are used in research from a variety of medical disciplines2,6,11,12 ; however, little is known about the understanding and use of statistics by pathology trainees and practicing pathologists. The specifics of statistical knowledge required to be an effective pathologist is also not known and probably varies by scope of practice. The objective of this study was 2-fold: (1) to determine the frequency of different statistical tests most commonly used in research studies published in pathology journals, and (2) to assess the degree of understanding pathologists profess to have of the most common statistical methods.
Literature Review on Reporting Frequency of Statistical Tests
We conducted a focused literature review of 1100 research articles to determine the types of statistical tests commonly reported in research articles published in pathology journals. We obtained and audited a random sample of 100 original articles per journal published during 2015 in 11 pathology journals: American Journal of Clinical Pathology (AJCP), The American Journal of Pathology (AJP), American Journal of Surgical Pathology (AJSP), Archives of Pathology & Laboratory Medicine (Archives), Cancer Cytopathology (CAC), Clinica Chimica Acta (CCA), Clinical Chemistry (CC), Diagnostic Cytopathology (DC), Journal of Pathology (JP), Laboratory Investigation (LI), and Modern Pathology (MP). These journals provide broad coverage of pathology and laboratory medicine. Small case series (<20 cases), reviews, and commentaries were excluded. Data were extracted by one author (R.L.S.). In some cases, we included articles from 2014 if the journal did not publish 100 original articles in 2015. Each article was reviewed to determine whether statistical tests were used and, if so, the specific techniques were recorded. Articles were classified as using no statistical methods, using descriptive methods, or using inferential methods. Articles were classified as using inferential methods if they reported P values but did not report the statistical test used. In such cases, we inferred the statistical test from the type of data; these cases were relatively rare (∼1%). An article was identified as using descriptive statistical methods if it used either graphical displays of data (eg, histograms, box and whisker charts, survival curves, or scatterplots) and/or calculated confidence intervals or interquartile ranges. Tabulation of numerical data, calculation of percentages, means, or medians (without distributional information) did not qualify as statistical analysis. We recorded the statistical tests used in each article as well as the statistical packages used. We classified journals into 5 categories: general (AJCP, Archives), basic science (AJP, JP, LI), translational (AJSP, MP), cytopathology (CAC, DC), and chemistry (CC, CCA).
Pathologist Survey on Knowledge of Statistical Tests
The Knowledge of Statistical Methods Survey was developed by 2 pathologists and reviewed by an educational expert to assess knowledge level of statistical techniques identified in the literature review by pathology faculty and trainees. We solicited pathology programs to participate in the survey by placing an announcement on a listserv for residency program directors. The following pathology programs agreed to participate in the study: Cleveland Clinic (Cleveland, Ohio), Icahn School of Medicine at Mount Sinai Medical Center (New York, New York), Thomas Jefferson University Hospital (Philadelphia, Pennsylvania), University of Missouri (Columbia), University of Utah (Salt Lake City), East Carolina University (Greenville, North Carolina), University of California at San Francisco (San Francisco), Loyola University (Maywood, Illinois), and Baylor College of Medicine (Houston, Texas). The survey was administered from the University of Utah by using Qualtrics software (Qualtrics LLC, Provo, Utah). The survey questionnaire is presented in the Supplemental Digital Content (supplemental digital content, containing the survey and 3 tables, is available for this article at www.archivesofpathology.org in the February 2017 table of contents). Potential respondents were provided by site coordinators at each site. Potential respondents were sent an invitation email with a Web link to the survey in January/February of 2016. Nonresponders were sent 2 reminder invitations at 1-week intervals. The study was approved by the Institutional Review Board at the University of Utah.
Statistical Training by Program
Admission requirements for statistics and calculus for all US medical schools were obtained from the Web site of the American Association of Medical Colleges (AAMC).13 Data on statistics training in medical school programs were also obtained from the AAMC. Program directors for each pathology residency program were surveyed regarding statistics and EBM training at each of the participating institutions.
Assessment of Statistical Knowledge
Knowledge of statistical methods was evaluated on a 3-point scale: 0 if the respondent had no useful knowledge of the test (had never heard of a method or only recognized the name), 1 if the respondent had basic knowledge (could interpret the test but was unable to perform it himself/herself), or 2 if the respondent had advanced knowledge (could perform and interpret the test himself/herself). A respondent statistical knowledge (RSK) variable was created by averaging knowledge of statistical methods ratings for each individual. The RSK score provided a measure of the statistical knowledge of an individual respondent averaged over all tests. The RSK score could range from 0 (no knowledge of any test) to 2 (able to perform and interpret all tests). Intermediate RSK scores are more difficult to interpret, but higher RSK scores indicate that a respondent has more skill with more tests. For example, an RSK score of 1.0 could mean that a respondent had basic knowledge (could interpret but not perform) of all tests, or that he/she had a range of knowledge that averaged out to 1.0 over all tests. We also created a test statistical knowledge (TSK) variable. The TSK score was calculated by averaging the responses for an individual test. The TSK score could range from 0 (all respondents indicated no knowledge of the test) to 2 (all respondents could perform the test and interpret the results without assistance).
Respondent statistical knowledge scores were compared by using the Kruskal-Wallis test. The χ2 test was used to evaluate the association between categorical variables. The correlation between TSK and test usage was evaluated by using the Spearman rank correlation coefficient. Statistical calculations were performed by using Stata 13 (Stata LLC, College Station, Texas).
Statistics Methods Used in the Research Literature
We abstracted data from 1100 original research articles (Table 1), of which 888 (81%) reported the use of statistical methods. Of the studies reporting statistical analysis, 770 of 888 (86%) used statistical tests (inferential statistics), and the remainder used descriptive statistics. The use of statistical methods ranged from 140 of 200 (70%) in general journals to 282 of 300 (94%) in basic science journals. Data comparing statistical test use in individual journals are presented in Supplemental Material Table 1. Among individual journals, statistical method use (descriptive or inferential) ranged from 61 of 100 (Modern Pathology) to 98 of 100 (American Journal of Pathology). There were significant differences in the pattern of test use among journal categories ( = 313, P < .001). That is, particular tests were used more frequently in different categories of journals. There was no significant difference in test usage among the journals within the general, translational, or cytology categories ( test, P > .43 for each category). There was a significant difference in test use by journals in the basic science and chemistry categories (P < .01).
We identified 18 statistical tests that appeared in at least 1% of studies (Table 1). The tests are described in Supplemental Material Table 2. The most frequently used tests were the t test (used in 297 of 1100 articles, or 27%), regression or analysis of variance (209 of 1100 articles, or 19%), χ2 test (198 of 1100 articles, or 18%), Mann-Whitney test (167 of 1100 articles, 15.2%), Fisher exact test (143 of 1100 articles, 13%), and tests for survival analysis (154 of 1100, or 14%).
Use of Statistical Software
Seven hundred twenty-six of 1100 articles (66%) reported the type of software used for statistical analysis. Twenty-nine different software packages were used. Statistical Program for the Social Sciences (SPSS; IBM Corp, Armonk, New York), GraphPad (GraphPad Software Inc, La Jolla, California), and Statistical Analysis System (SAS; SAS Institute Inc, Cary, North Carolina) were the most commonly used software programs (Table 2). Data comparing statistical software use in individual journals are presented in Supplemental Material Table 3.
We surveyed faculty and trainees from 9 pathology residency programs regarding their knowledge of statistical methods. Program directors at each site provided an email list of potential targets for the survey (faculty and trainees). This produced a list of 879 potential respondents (580 faculty and 299 trainees) who were sent the survey. The overall response rate to the survey was 365 of 879 (42%). The trainee response rate (136 of 299, 45%) was not significantly greater () than the faculty response rate (229 of 580, 39%) (Table 3).
Characteristics of Respondents
Ninety-seven percent of respondents indicated that they were MD/DO trained. Of the 365 who responded to the survey, 229 (63%) were faculty, and 136 (37%) were trainees (Table 3). One hundred twelve (56%) were primarily focused on anatomic pathology, 64 (32%) were focused on clinical pathology, and 25 (12%) indicated that their work was balanced between the two. One hundred twenty-seven respondents (41%) indicated that they had earned an advanced degree in addition to the MD degree. Of these, 95 (72%) had earned a PhD degree. Faculty respondents were evenly distributed across the categories for years in practice. Two hundred sixty-four of 321 respondents (82%) indicated that they conducted research studies for publication; 223 of 309 (73%) said that publication was required for professional advancement. Respondents differed in their use of statistics. Of 314 respondents, 208 (66%) indicated that they mainly needed statistics for research, 57 (18%) primarily used statistics for clinical work or operations, and 49 (16%) indicated that they had no need for statistics in their work. More than half of respondents (157 of 265, 59%) said that they rely on a professional statistician or a knowledgeable colleague to perform statistical analyses, while 92 (35%) perform statistical analyses themselves. The respondents had a wide range of research experience. For trainees, the median number of published articles was 5 (range: 0–45, interquartile range: 2–10). For faculty, the median number of published articles was 32 (range: 2–380, interquartile range: 16–91). Almost all respondents (302 of 310, 97%) indicated that a better understanding of statistics would be helpful.
Statistical training can occur at several points during training (before medical school, during medical school, during residency or fellowship, or from other graduate training) or after training through self-study and experience. Fifty-nine percent of respondents had taken a statistics course at some point in their career. Having had a statistics course was strongly associated with having an advanced degree other than an MD degree (. Only 6 of 148 medical schools (4%) list a course in statistics as an admission requirement, and only 34 of 148 (23%) recommend a course in statistics before enrollment. The distribution of requirements for calculus (14 schools require, 31 recommend) was not statistically different from the distribution of requirements for statistics (. In 2013, most medical schools reported that they required their medical students to complete a biostatistics course (140 of 154), a course in clinical and translational research (134 of 154), an epidemiology course (139 of 154), a course in literature evaluation (137 of 154), a course in EBM (140 of 154), or a course in research methods (140 of 154). Our sample of residency programs showed wide variation in statistical training (Table 4). Statistical training is mostly informal (journal club, rotations), but some programs provide formal lectures.
Knowledge of Statistical Methods
The average TSK score (average knowledge for all respondents for a particular test) was 0.70 and ranged from 0.3 (Kruskal-Wallis test) to 1.33 (Student t test). The TSK scores data are summarized in Table 5. The median RSK score was 0.56 (interquartile range: 0.25–1.06; Figure 1, A through C). Overall, respondents' knowledge of tests correlated with the frequency with which tests are used in journal articles (Spearman ρ = 0.74, P = .003; Figure 2). For example, respondents reported the greatest knowledge of the χ2 test and Student t test, which were also the most widely used methods in the examined journals (Table 5, Figure 2).
Factors Associated With Knowledge of Statistical Tests
The factors associated with statistical knowledge are summarized in Table 6. There was no significant difference in the RSK scores between trainees and faculty (P = .14), and there was no association between RSK scores and years in practice (P = .20). Greater statistical knowledge was associated with a focus in clinical pathology rather than anatomic pathology (P < .001), having an advanced degree other than an MD degree (P < .001), having taken a statistics course (P < .01), and conducting research studies for publication (P < .001). The type of advanced degree (PhD versus master's level) had a borderline association with statistical knowledge (P = .06). Trainees at programs with formal statistics training had higher RSK scores than those without (Kruskal-Wallis test, P = .03).
A list of statistical tests that are frequently used in the pathology literature provides a starting point for understanding the type of statistical knowledge that is useful for pathologists, and provides a direction for how to train pathologists in statistics and EBM. Our study found that a small number of statistical tests are frequently used in pathology studies. A working knowledge of approximately 12 tests would enable a pathologist to understand the statistical methods used in most research published in pathology journals.
The frequency of test use that we report here is similar to what was reported in a previous study focused on family and emergency medicine,11 as well as a recent study of cytopathology journals.14 However, pathology is a varied discipline, and our study discovered that different categories of pathology journals show significant differences in the type of statistical tests that they commonly report. For example, basic science and chemistry journals tend to use analysis of variance and regression, translational journals tend to use survival methods and κ statistics, and cytopathology journals tend to use categorical methods. This suggests that optimal statistical training of pathologists may need to be focused on different areas of specialization.
Our survey found that familiarity of tests by pathology faculty and trainees highly correlated with the frequency of tests commonly reported in the literature. Our RSK scale is relative and does not provide an absolute measurement of statistical literacy, but our findings suggest that statistical literacy is lacking for trainees and faculty alike. As indicated in Table 5, most pathologists can interpret but not perform simple tests such as the χ2 and t tests. Our survey found that most pathologists (59%) solicit the help of a knowledgeable colleague or a professional statistician to perform statistical tests. While pathologists may only need basic knowledge because they have the benefit of working with a statistician, the fact that 41% of respondents did not indicate that they work with a statistician suggests that pathologists may not have access to statisticians at some academic institutions. This is worth exploring further, because not only is there a lack of knowledge of statistics, but also potentially a lack of resources for pathologists to use statistical experts. Moreover, 96% of the respondents indicated that having a greater knowledge of statistics would be beneficial, so while a basic knowledge of statistics may be sufficient for most work roles, it may not be sufficient for particular roles such as teaching EBM, reviewing papers, conducting studies, or designing quality improvement experiments.
Statistical knowledge is not limited to understanding methods. It also requires an appreciation for broader statistical ideas such as hypothesis testing, statistical power, key assumptions, and differences between parametric and nonparametric methods. At a broader level, there are key concepts related to study design and good statistical practice (eg, reporting guidelines, reproducibility). Pathologists do not need to know a long list of methods; however, knowledge of a few provides a bridge to understanding the broader conceptual issues that could help pathologists work effectively with statisticians or colleagues who have statistical knowledge. Pathologists need a basic-level understanding of general statistical concepts, and this is enhanced by an advanced-level knowledge of at least a few tests.
We found that pathologists do not acquire statistical skills through lifelong learning. Statistical knowledge was not associated with years of experience or with research experience (estimated by number of publications). This suggests that statistical knowledge is mostly developed during the training years. When we evaluated statistics requirements for medical school, we found that relatively few schools require statistics for admission. Our survey did not assess the percentage of medical students who took an undergraduate course in statistics, an area for further research. Almost all medical schools provide training in EBM and statistics; however, the format, extent, and effectiveness of medical school training are not standardized. Our survey showed that statistical knowledge is highly associated with an advanced degree in another discipline, and that statistical knowledge is associated with taking a formal course in statistics. Thus, the statistical knowledge of pathologists obtained only in medical school and residency is likely to be inadequate.
Almost all survey respondents indicated that they would benefit from more statistical training. For residents, statistics training is often delivered via journal clubs. Although journal clubs may be an effective mechanism for developing high-level conceptual knowledge such as critical appraisal and study design, it may not be an ideal method for developing statistical knowledge and skills. Our results suggest that formal lectures may be more effective than informal (and nonstandardized) avenues such as journal club or discussion on rotations.
Given that RSK scores of faculty did not differ from those of trainees, it is likely that many programs would not have faculty with the expertise to provide training in statistics. Developing statistical training material would require substantial duplication of effort if each training program developed such materials independently. Although online courses are available (eg, via Coursera, edX, Udacity, statistics.com), these courses may not be tailored to the needs of pathology trainees. Thus, it would be helpful if pathology organizations (eg, American Society for Clinical Pathology, College of American Pathologists) pooled resources to develop training materials that could be shared among programs. A Web-based pathology journal club, such as the General Surgery Journal Club, might also provide a cost-effective way to pool resources and avoid duplication of effort.15 Such training materials could benefit both trainees and practicing pathologists.
To our knowledge, ours is the first study to investigate statistical literacy of pathologists by using a survey. It should be noted that this method has inherent limitations. Our survey of statistical knowledge was based on self-reported knowledge rather than an objective test, which we felt would be practically infeasible. Our list of statistical tests may have been limited by the journals we selected, though we attempted to capture a wide range of journals providing broad coverage of pathology and laboratory medicine. Pathology is a large and varied discipline, and we realize the methods reported in research journals may not reflect the statistical methods required for routine clinical work or operations improvement. Similarly, pathologists do read clinical medical journals, and our study did not assess the methods used in such journals. We also only assessed methods used in original research articles, though many other types of articles (eg, reviews, case studies, commentaries) are useful to pathologists. Our study did not evaluate whether methods were used appropriately; we only tabulated reported methods. Some tests may have been used but not reported, so our estimate of frequency is most likely an underestimate. Our rank order of tests is probably accurate unless the number of unreported tests is high and the distribution of unreported tests is quite different from reported tests. Our response rate was 42%; thus, the respondents to the survey may not reflect the overall population of pathologists and residents. And lastly, we only surveyed pathologists at academic institutions, so we cannot gauge the knowledge of statistics by pathologists in private practice.
Finally, the need for statistical literacy among pathologists is likely to grow. New types of research (genomics, health services research, computational pathology) will require greater statistical knowledge, and statistical knowledge is required for quality improvement and assurance initiatives, which are now required for maintenance of certification. Overall, there appears to be an unmet need for statistical training among pathologists.
In conclusion, despite its limitations, our survey has pinpointed a number of key findings that warrant further investigation. In particular, even those pathologists who professed to have some understanding of statistics responded that having greater knowledge of statistics would be beneficial. Because medical training is so long and often circuitous for those acquiring additional degrees, residency may be a good place to start with the foundations of statistical literacy, but there is also a need to offer additional training and reinforcement of statistical concepts to practicing pathologists.
Supplemental digital content is available for this article at www.archivesofpathology.org in the February 2017 table of contents.
From the Department of Pathology, University of Utah Health Sciences Center, and ARUP Laboratories, Salt Lake City, Utah (Drs Schmidt, Smock, and Factor); the Department of Pathology, Cleveland Clinic Lerner College of Medicine, Cleveland, Ohio (Dr Chute); the Department of Internal Medicine, University of Utah, Salt Lake City (Dr Colbert-Getz); the Department of Pathology, Icahn School of Medicine at Mount Sinai, New York, New York (Dr Firpo-Betancourt); the Departments of Marketing (Mr James) and Informatics (Mr Walker), ARUP Laboratories, Salt Lake City, Utah; the Department of Pathology, Anatomy, and Cell Biology, Thomas Jefferson University Hospital, Philadelphia, Pennsylvania (Dr Karp); the Department of Pathology & Anatomical Sciences, University of Missouri, Columbia (Dr Miller); the Department of Pathology, Brigham and Women's Hospital, Boston, Massachusetts (Dr Milner); the Department of Pathology and Laboratory Medicine, Brody School of Medicine, East Carolina University, Greenville, North Carolina (Dr Sutton); the Department of Laboratory Medicine, University of California at San Francisco, San Francisco (Dr White); College of Nursing, Health Sciences Center, University of Utah, Salt Lake City (Dr Wilson); the Department of Pathology, Loyola University, Chicago, Illinois (Dr Wojcik); and the Department of Pathology, Baylor College of Medicine, Houston, Texas (Dr Yared).
The authors have no relevant financial interest in the products or companies described in this article.