Statistical Literacy Among Academic Pathologists: A Survey Study to Gauge Knowledge of Frequently Used Statistical Tests Among Trainees and Faculty

We conducted a focused literature review of 1100 research articles to determine the types of statistical tests commonly reported in research articles published in pathology journals. We obtained and audited a random sample of 100 original articles per journal published during 2015 in 11 pathology journals: American Journal of Clinical Pathology (AJCP), The American Journal of Pathology (AJP), American Journal of Surgical Pathology (AJSP), Archives of Pathology & Laboratory Medicine (Archives), Cancer Cytopathology (CAC), Clinica Chimica Acta (CCA), Clinical Chemistry (CC), Diagnostic Cytopathology (DC), Journal of Pathology (JP), Laboratory Investigation (LI), and Modern Pathology (MP). These journals provide broad coverage of pathology and laboratory medicine. Small case series (<20 cases), reviews, and commentaries were excluded. Data were extracted by one author (R.L.S.). In some cases, we included articles from 2014 if the journal did not publish 100 original articles in 2015. Each article was reviewed to determine whether statistical tests were used and, if so, the specific techniques were recorded. Articles were classified as using no statistical methods, using descriptive methods, or using inferential methods. Articles were classified as using inferential methods if they reported P values but did not report the statistical test used. In such cases, we inferred the statistical test from the type of data; these cases were relatively rare (∼1%). An article was identified as using descriptive statistical methods if it used either graphical displays of data (eg, histograms, box and whisker charts, survival curves, or scatterplots) and/or calculated confidence intervals or interquartile ranges. Tabulation of numerical data, calculation of percentages, means, or medians (without distributional information) did not qualify as statistical analysis. We recorded the statistical tests used in each article as well as the statistical packages used. We classified journals into 5 categories: general (AJCP, Archives), basic science (AJP, JP, LI), translational (AJSP, MP), cytopathology (CAC, DC), and chemistry (CC, CCA).

The Knowledge of Statistical Methods Survey was developed by 2 pathologists and reviewed by an educational expert to assess knowledge level of statistical techniques identified in the literature review by pathology faculty and trainees. We solicited pathology programs to participate in the survey by placing an announcement on a listserv for residency program directors. The following pathology programs agreed to participate in the study: Cleveland Clinic (Cleveland, Ohio), Icahn School of Medicine at Mount Sinai Medical Center (New York, New York), Thomas Jefferson University Hospital (Philadelphia, Pennsylvania), University of Missouri (Columbia), University of Utah (Salt Lake City), East Carolina University (Greenville, North Carolina), University of California at San Francisco (San Francisco), Loyola University (Maywood, Illinois), and Baylor College of Medicine (Houston, Texas). The survey was administered from the University of Utah by using Qualtrics software (Qualtrics LLC, Provo, Utah). The survey questionnaire is presented in the Supplemental Digital Content (supplemental digital content, containing the survey and 3 tables, is available for this article at www.archivesofpathology.org in the February 2017 table of contents). Potential respondents were provided by site coordinators at each site. Potential respondents were sent an invitation email with a Web link to the survey in January/February of 2016. Nonresponders were sent 2 reminder invitations at 1-week intervals. The study was approved by the Institutional Review Board at the University of Utah.

Admission requirements for statistics and calculus for all US medical schools were obtained from the Web site of the American Association of Medical Colleges (AAMC).¹³  Data on statistics training in medical school programs were also obtained from the AAMC. Program directors for each pathology residency program were surveyed regarding statistics and EBM training at each of the participating institutions.

Knowledge of statistical methods was evaluated on a 3-point scale: 0 if the respondent had no useful knowledge of the test (had never heard of a method or only recognized the name), 1 if the respondent had basic knowledge (could interpret the test but was unable to perform it himself/herself), or 2 if the respondent had advanced knowledge (could perform and interpret the test himself/herself). A respondent statistical knowledge (RSK) variable was created by averaging knowledge of statistical methods ratings for each individual. The RSK score provided a measure of the statistical knowledge of an individual respondent averaged over all tests. The RSK score could range from 0 (no knowledge of any test) to 2 (able to perform and interpret all tests). Intermediate RSK scores are more difficult to interpret, but higher RSK scores indicate that a respondent has more skill with more tests. For example, an RSK score of 1.0 could mean that a respondent had basic knowledge (could interpret but not perform) of all tests, or that he/she had a range of knowledge that averaged out to 1.0 over all tests. We also created a test statistical knowledge (TSK) variable. The TSK score was calculated by averaging the responses for an individual test. The TSK score could range from 0 (all respondents indicated no knowledge of the test) to 2 (all respondents could perform the test and interpret the results without assistance).

Respondent statistical knowledge scores were compared by using the Kruskal-Wallis test. The χ² test was used to evaluate the association between categorical variables. The correlation between TSK and test usage was evaluated by using the Spearman rank correlation coefficient. Statistical calculations were performed by using Stata 13 (Stata LLC, College Station, Texas).

We abstracted data from 1100 original research articles (Table 1), of which 888 (81%) reported the use of statistical methods. Of the studies reporting statistical analysis, 770 of 888 (86%) used statistical tests (inferential statistics), and the remainder used descriptive statistics. The use of statistical methods ranged from 140 of 200 (70%) in general journals to 282 of 300 (94%) in basic science journals. Data comparing statistical test use in individual journals are presented in Supplemental Material Table 1. Among individual journals, statistical method use (descriptive or inferential) ranged from 61 of 100 (Modern Pathology) to 98 of 100 (American Journal of Pathology). There were significant differences in the pattern of test use among journal categories (⁠ = 313, P < .001). That is, particular tests were used more frequently in different categories of journals. There was no significant difference in test usage among the journals within the general, translational, or cytology categories (⁠ test, P > .43 for each category). There was a significant difference in test use by journals in the basic science and chemistry categories (P < .01).

We identified 18 statistical tests that appeared in at least 1% of studies (Table 1). The tests are described in Supplemental Material Table 2. The most frequently used tests were the t test (used in 297 of 1100 articles, or 27%), regression or analysis of variance (209 of 1100 articles, or 19%), χ² test (198 of 1100 articles, or 18%), Mann-Whitney test (167 of 1100 articles, 15.2%), Fisher exact test (143 of 1100 articles, 13%), and tests for survival analysis (154 of 1100, or 14%).

Seven hundred twenty-six of 1100 articles (66%) reported the type of software used for statistical analysis. Twenty-nine different software packages were used. Statistical Program for the Social Sciences (SPSS; IBM Corp, Armonk, New York), GraphPad (GraphPad Software Inc, La Jolla, California), and Statistical Analysis System (SAS; SAS Institute Inc, Cary, North Carolina) were the most commonly used software programs (Table 2). Data comparing statistical software use in individual journals are presented in Supplemental Material Table 3.

We surveyed faculty and trainees from 9 pathology residency programs regarding their knowledge of statistical methods. Program directors at each site provided an email list of potential targets for the survey (faculty and trainees). This produced a list of 879 potential respondents (580 faculty and 299 trainees) who were sent the survey. The overall response rate to the survey was 365 of 879 (42%). The trainee response rate (136 of 299, 45%) was not significantly greater (⁠⁠) than the faculty response rate (229 of 580, 39%) (Table 3).

Ninety-seven percent of respondents indicated that they were MD/DO trained. Of the 365 who responded to the survey, 229 (63%) were faculty, and 136 (37%) were trainees (Table 3). One hundred twelve (56%) were primarily focused on anatomic pathology, 64 (32%) were focused on clinical pathology, and 25 (12%) indicated that their work was balanced between the two. One hundred twenty-seven respondents (41%) indicated that they had earned an advanced degree in addition to the MD degree. Of these, 95 (72%) had earned a PhD degree. Faculty respondents were evenly distributed across the categories for years in practice. Two hundred sixty-four of 321 respondents (82%) indicated that they conducted research studies for publication; 223 of 309 (73%) said that publication was required for professional advancement. Respondents differed in their use of statistics. Of 314 respondents, 208 (66%) indicated that they mainly needed statistics for research, 57 (18%) primarily used statistics for clinical work or operations, and 49 (16%) indicated that they had no need for statistics in their work. More than half of respondents (157 of 265, 59%) said that they rely on a professional statistician or a knowledgeable colleague to perform statistical analyses, while 92 (35%) perform statistical analyses themselves. The respondents had a wide range of research experience. For trainees, the median number of published articles was 5 (range: 0–45, interquartile range: 2–10). For faculty, the median number of published articles was 32 (range: 2–380, interquartile range: 16–91). Almost all respondents (302 of 310, 97%) indicated that a better understanding of statistics would be helpful.

Statistical training can occur at several points during training (before medical school, during medical school, during residency or fellowship, or from other graduate training) or after training through self-study and experience. Fifty-nine percent of respondents had taken a statistics course at some point in their career. Having had a statistics course was strongly associated with having an advanced degree other than an MD degree (⁠⁠. Only 6 of 148 medical schools (4%) list a course in statistics as an admission requirement, and only 34 of 148 (23%) recommend a course in statistics before enrollment. The distribution of requirements for calculus (14 schools require, 31 recommend) was not statistically different from the distribution of requirements for statistics (⁠⁠. In 2013, most medical schools reported that they required their medical students to complete a biostatistics course (140 of 154), a course in clinical and translational research (134 of 154), an epidemiology course (139 of 154), a course in literature evaluation (137 of 154), a course in EBM (140 of 154), or a course in research methods (140 of 154). Our sample of residency programs showed wide variation in statistical training (Table 4). Statistical training is mostly informal (journal club, rotations), but some programs provide formal lectures.

The average TSK score (average knowledge for all respondents for a particular test) was 0.70 and ranged from 0.3 (Kruskal-Wallis test) to 1.33 (Student t test). The TSK scores data are summarized in Table 5. The median RSK score was 0.56 (interquartile range: 0.25–1.06; Figure 1, A through C). Overall, respondents' knowledge of tests correlated with the frequency with which tests are used in journal articles (Spearman ρ = 0.74, P = .003; Figure 2). For example, respondents reported the greatest knowledge of the χ² test and Student t test, which were also the most widely used methods in the examined journals (Table 5, Figure 2).

The factors associated with statistical knowledge are summarized in Table 6. There was no significant difference in the RSK scores between trainees and faculty (P = .14), and there was no association between RSK scores and years in practice (P = .20). Greater statistical knowledge was associated with a focus in clinical pathology rather than anatomic pathology (P < .001), having an advanced degree other than an MD degree (P < .001), having taken a statistics course (P < .01), and conducting research studies for publication (P < .001). The type of advanced degree (PhD versus master's level) had a borderline association with statistical knowledge (P = .06). Trainees at programs with formal statistics training had higher RSK scores than those without (Kruskal-Wallis test, P = .03).

A list of statistical tests that are frequently used in the pathology literature provides a starting point for understanding the type of statistical knowledge that is useful for pathologists, and provides a direction for how to train pathologists in statistics and EBM. Our study found that a small number of statistical tests are frequently used in pathology studies. A working knowledge of approximately 12 tests would enable a pathologist to understand the statistical methods used in most research published in pathology journals.

The frequency of test use that we report here is similar to what was reported in a previous study focused on family and emergency medicine,¹¹  as well as a recent study of cytopathology journals.¹⁴  However, pathology is a varied discipline, and our study discovered that different categories of pathology journals show significant differences in the type of statistical tests that they commonly report. For example, basic science and chemistry journals tend to use analysis of variance and regression, translational journals tend to use survival methods and κ statistics, and cytopathology journals tend to use categorical methods. This suggests that optimal statistical training of pathologists may need to be focused on different areas of specialization.

Our survey found that familiarity of tests by pathology faculty and trainees highly correlated with the frequency of tests commonly reported in the literature. Our RSK scale is relative and does not provide an absolute measurement of statistical literacy, but our findings suggest that statistical literacy is lacking for trainees and faculty alike. As indicated in Table 5, most pathologists can interpret but not perform simple tests such as the χ² and t tests. Our survey found that most pathologists (59%) solicit the help of a knowledgeable colleague or a professional statistician to perform statistical tests. While pathologists may only need basic knowledge because they have the benefit of working with a statistician, the fact that 41% of respondents did not indicate that they work with a statistician suggests that pathologists may not have access to statisticians at some academic institutions. This is worth exploring further, because not only is there a lack of knowledge of statistics, but also potentially a lack of resources for pathologists to use statistical experts. Moreover, 96% of the respondents indicated that having a greater knowledge of statistics would be beneficial, so while a basic knowledge of statistics may be sufficient for most work roles, it may not be sufficient for particular roles such as teaching EBM, reviewing papers, conducting studies, or designing quality improvement experiments.

Statistical knowledge is not limited to understanding methods. It also requires an appreciation for broader statistical ideas such as hypothesis testing, statistical power, key assumptions, and differences between parametric and nonparametric methods. At a broader level, there are key concepts related to study design and good statistical practice (eg, reporting guidelines, reproducibility). Pathologists do not need to know a long list of methods; however, knowledge of a few provides a bridge to understanding the broader conceptual issues that could help pathologists work effectively with statisticians or colleagues who have statistical knowledge. Pathologists need a basic-level understanding of general statistical concepts, and this is enhanced by an advanced-level knowledge of at least a few tests.

We found that pathologists do not acquire statistical skills through lifelong learning. Statistical knowledge was not associated with years of experience or with research experience (estimated by number of publications). This suggests that statistical knowledge is mostly developed during the training years. When we evaluated statistics requirements for medical school, we found that relatively few schools require statistics for admission. Our survey did not assess the percentage of medical students who took an undergraduate course in statistics, an area for further research. Almost all medical schools provide training in EBM and statistics; however, the format, extent, and effectiveness of medical school training are not standardized. Our survey showed that statistical knowledge is highly associated with an advanced degree in another discipline, and that statistical knowledge is associated with taking a formal course in statistics. Thus, the statistical knowledge of pathologists obtained only in medical school and residency is likely to be inadequate.

Almost all survey respondents indicated that they would benefit from more statistical training. For residents, statistics training is often delivered via journal clubs. Although journal clubs may be an effective mechanism for developing high-level conceptual knowledge such as critical appraisal and study design, it may not be an ideal method for developing statistical knowledge and skills. Our results suggest that formal lectures may be more effective than informal (and nonstandardized) avenues such as journal club or discussion on rotations.

Given that RSK scores of faculty did not differ from those of trainees, it is likely that many programs would not have faculty with the expertise to provide training in statistics. Developing statistical training material would require substantial duplication of effort if each training program developed such materials independently. Although online courses are available (eg, via Coursera, edX, Udacity, statistics.com), these courses may not be tailored to the needs of pathology trainees. Thus, it would be helpful if pathology organizations (eg, American Society for Clinical Pathology, College of American Pathologists) pooled resources to develop training materials that could be shared among programs. A Web-based pathology journal club, such as the General Surgery Journal Club, might also provide a cost-effective way to pool resources and avoid duplication of effort.¹⁵  Such training materials could benefit both trainees and practicing pathologists.

To our knowledge, ours is the first study to investigate statistical literacy of pathologists by using a survey. It should be noted that this method has inherent limitations. Our survey of statistical knowledge was based on self-reported knowledge rather than an objective test, which we felt would be practically infeasible. Our list of statistical tests may have been limited by the journals we selected, though we attempted to capture a wide range of journals providing broad coverage of pathology and laboratory medicine. Pathology is a large and varied discipline, and we realize the methods reported in research journals may not reflect the statistical methods required for routine clinical work or operations improvement. Similarly, pathologists do read clinical medical journals, and our study did not assess the methods used in such journals. We also only assessed methods used in original research articles, though many other types of articles (eg, reviews, case studies, commentaries) are useful to pathologists. Our study did not evaluate whether methods were used appropriately; we only tabulated reported methods. Some tests may have been used but not reported, so our estimate of frequency is most likely an underestimate. Our rank order of tests is probably accurate unless the number of unreported tests is high and the distribution of unreported tests is quite different from reported tests. Our response rate was 42%; thus, the respondents to the survey may not reflect the overall population of pathologists and residents. And lastly, we only surveyed pathologists at academic institutions, so we cannot gauge the knowledge of statistics by pathologists in private practice.

Finally, the need for statistical literacy among pathologists is likely to grow. New types of research (genomics, health services research, computational pathology) will require greater statistical knowledge, and statistical knowledge is required for quality improvement and assurance initiatives, which are now required for maintenance of certification. Overall, there appears to be an unmet need for statistical training among pathologists.

In conclusion, despite its limitations, our survey has pinpointed a number of key findings that warrant further investigation. In particular, even those pathologists who professed to have some understanding of statistics responded that having greater knowledge of statistics would be beneficial. Because medical training is so long and often circuitous for those acquiring additional degrees, residency may be a good place to start with the foundations of statistical literacy, but there is also a need to offer additional training and reinforcement of statistical concepts to practicing pathologists.