Context.—To avoid errors many surgical pathology services mandate review of a case by a second pathologist before reports are released (signed out).
Objective.—To study the extent and characteristics of such review.
Design.—Participants retrospectively examined up to 400 cases to identify a maximum of 30 cases reviewed by at least one additional pathologist before sign-out. For each case, participants documented the organ system, primary disease type, number of additional pathologists consulted, and the reason for case review. The main outcome measure was the fraction of surgical pathology cases that underwent second pathologist review before sign-out.
Results.—From 45 laboratories, examination of 18 032 surgical pathology cases yielded 1183 (6.6%) cases that had been reviewed before sign-out. The median laboratory reviewed 8.2% of cases. Three-fifths of reviews focused on 4 organ systems: gastrointestinal (20.5%), breast (16.0%), skin (12.7%), and female genital tract (10.0%). Malignant neoplasm far exceeded all other categories of disease in reviewed cases (45.3%). Cases were reviewed by one additional pathologist 78% of the time. Two dominant reasons for case review emerged: difficult diagnosis (46.2%) and audit required by departmental policy (43.0%). Most laboratories (71%) had departmental policies regarding review of cases. These laboratories reviewed cases about 33% more often than laboratories without policies (9.6% versus 6.5%).
Conclusions.—Review of selected surgical pathology cases before sign-out is widely accepted with 71% of participant laboratories following policies to this effect. About 1 case in 15 (6.6%) were reviewed with the median laboratory of participants reviewing about 1 in 12 (8.2%).
Early in training, pathologists are taught to seek the opinion of others when confronted with difficult diagnostic cases. Most pathologists develop particular interest and knowledge in specific organs and tend to seek the advice of others when presented with challenging cases outside this domain of interest and knowledge. Until recently, seeking advice on cases was a voluntary exercise. Recent attention to diagnostic errors in the medical literature and the public press has led to many pathology practices standardizing such case reviews as a plausible measure to prevent errors and to provide assurance of diagnostic accuracy.1–11
This trend has often been adopted through policies that require second pathologist review of particular types of case. The content of the case review tends to reflect previous specific experience with misdiagnosis in each surgical pathology practice. We report here a Q-Probes study of practices in 45 surgical pathology laboratories that reviewed cases before they were released (review before sign-out). To our knowledge, this is the first study to measure the frequency and characteristics of case review before sign-out in multiple institutions.
MATERIALS AND METHODS
This study was offered and conducted as a Q-Probes study, the basic mechanism of which has been previously described.12 Briefly, Q-Probes studies are subscription-based quality assurance studies that are conducted at a certain point in time and have the advantage of peer group comparison to assist in determining level of performance.
This study was offered and conducted in the fourth quarter of 2008. Participants examined accessioned cases retrospectively from August 1, 2008, until 30 surgical pathology cases were identified that had documentation of review by at least a second pathologist before sign-out or until 400 consecutively accessioned cases had been examined.
For each identified, reviewed case, participants documented the organ system and primary disease type involved in the diagnosis, the number of additional pathologists consulted, and the reason for case review. Cases reviewed after sign-out and cases for which the second reviewer was a resident or fellow were excluded from the study. Participant laboratories also provided demographic information describing their laboratory and the institutions at which they serve, as well as practice variables that the study authors hypothesized might be linked to surgical pathology case review.
When data distributions for case review rate were statistically analyzed, the review rate was slightly skewed toward higher review rates, so a rank-based analysis for the univariate tests and log-transformation for the multivariate regression analysis were used. One participant had a 0% review rate and was excluded from the statistical analysis.
Individual associations between the review rate and the demographic and practice variables were investigated using Kruskal-Wallis tests for discrete-valued, independent variables and regression analysis for continuous, independent variables. Variables with significant associations (P < .10) were then introduced into a forward-selection, multivariate regression model. Variables remaining after this procedure would be significantly associated at the P < .05 significance level.
Of the 45 participant surgical pathology laboratories, most (89%) were located in the United States, 3 (7%) were in Saudi Arabia, and 1 (2%) each were in Australia and Canada; 37.8% (17 of 45) considered their laboratories to be teaching hospitals, and 22.2% (10 of 45) maintained pathology residency programs. Most of the laboratories (82.2%; 37 of 45) had been inspected by the Laboratory Accreditation program of the College of American Pathologists within 2 years of the study and 34.1% of institutions (15 of 44) had been inspected by the Joint Commission within the previous 2 years. Table 1 shows institutional demographic information about institutional type, bed size, location, and governmental affiliation. Table 2 shows the distribution of surgical pathology accessioned cases and of full-time pathologists who evaluated cases. The median participant laboratory processed 10 422 cases in 2007 and had 4 surgical pathologists as staff. Table 3 shows the characteristics that we hypothesized might influence case reviews; 32 of 45 laboratories (71.1%) had a policy regarding review of cases before case sign-out. Although most laboratories (24 of 45; 53%) did not target any percentage for review, 7 laboratories (15.6%) targeted 10% of cases for review, and 1 laboratory (2%) targeted 95% of cases.
In aggregate, 1183 cases (6.6%) were reviewed among 18 032 cases examined. The median rate of review was 8.2% with a skewed distribution shown in Table 4. The characteristics of the study sample, the number of cases reviewed per participating laboratory, and the number of accessioned cases examined to find the study sample of reviewed cases is shown in Table 5. Most participants reached the study target of 30 cases in less than 400 cases examined. Table 6 shows the organ systems involved in the cases reviewed before sign-out. The 4 most frequent categories, gastrointestinal, breast, skin, and female genital tract cases, accounted for about 60% of reviewed cases. Table 7 shows the primary disease type, number of pathologists consulted, and the reason for case review before sign-out. As expected, malignant neoplasms (45.3%) were the highest proportion of cases. When combined with premalignant (7.4%), low malignant/borderline neoplasm (2.8%), and malignant mimics (benign proliferation/hyperplasia/atypia) (15.7%), this accounted for nearly three-quarters of all cases.
In evaluating practice and demographic variables, 4 factors were significantly associated with the level of review. Laboratory policy regarding case reviews before sign-out was associated with a higher percentage of reviewed cases (9.6% versus 6.5%). Nonfederal governmental institutions had a significantly lower rate of review versus nongovernmental and federal governmental facilities (3.5% versus 8.4% and 12.4%, respectively). Laboratories with review policies also reviewed malignancies at a higher frequency than laboratories without such policies (P < .001; 48% versus 36% of reviewed cases). Laboratories with review policies also tended to review breast cases at a higher rate than those that did not (P = .003; median rate of 1.2% versus 0.5% of all cases). No such statistical significance was found with other organ systems.
To our knowledge, this is the first multi-institutional study to measure the extent and characteristics of pathology case review before case sign-out. The median laboratory reviewed 8.2% of cases with an aggregate rate of 6.6%. One other publication has attempted to measure the rate of case review before sign-out. Renshaw and Gould13 measured their institutional rate at 13%.
In our study, the range of the rate of case review was wide, laboratories in the 10th and 90th percentiles reviewed 2.0% and 17.1% of cases. This, however, likely represents a low estimate of the rate of intradepartmental consultation. Because this study only captures documented reviews, we can speculate that a number of cases are routinely shown to other pathologists but are not documented and, therefore, are not captured by a retrospective case review. In a previous Q-Probes study14 of expert extradepartmental consultation, a median rate of 0.5% was recorded with a range of 0% to 2.0%. In that study, expert extradepartmental consultations only represented consultations that were initiated by the pathologists and did not include cases that are seen at other institutions after sign-out because the patient seeks another opinion or goes to another institution for definitive treatment. The rate described in this report represents the prevalent rate of documented cases reviewed internally before sign-out.
Several quality-assurance guidelines have proposed a 10% review rate as a standard, without demonstrating the effectiveness of this specific rate of review.15,16 These mandated quality-assurance reviews are typically performed after cases are signed out and are used to evaluate error rates of an individual or a program.17 So they differ in timing—after sign-out versus before sign-out—and in intention—error detection versus error prevention—from the double-review practice examined in our study. Several studies have hypothesized that review of cases before sign-out reduces error and amended report rates.13,18,19 Our study did not collect error or amended report rates, so it cannot speak to that hypothesis. If potential errors can be identified before reports are issued, this preventative measure would indeed be an important quality measure. It is surprising to the authors of this study that the effectiveness of this procedure has not been tested. The appropriate rate and pattern of such reviews has not been empirically determined. Our study at least offers a picture of the extent of the practice at present.
Regarding patterns of review, we determined what types of cases were reviewed by study participants. The top 4 organ systems reviewed were gastrointestinal tract (20%), breast (16%), skin (13%), and the female genital tract (10%). Although the proportion of cases seen in a particular laboratory varies depending on the practice setting, GI, skin, and female genital tract specimens tend to be among the highest proportion of cases in most North American practice settings. In general practice, breast specimens tend to account for a small proportion of cases but are probably reviewed at a much higher level than other types of specimens because of the clinical implication of breast biopsy results. This is borne out in our statistical analysis where breast cases were reviewed at a higher median rate (1.2%) in laboratories with a review policy than in laboratories that did not have such a policy (0.5%). With respect to disease type, malignancy accounted for 45% of all reviewed cases and far exceeded any other category by a margin of 3 to 1. Selection of known malignancies for review is easy, but this review criterion leaves aside the issue of detecting cases of neoplasms that potentially could have been missed. These may have been misplaced in the second-most frequent category of benign neoplasm/hyperplasia/atypia. False-negative cases indeed represent the highest proportion (73% and 61%) of cases in 2 reports of legal verdict and settlements against pathologists.20,21 Some laboratories have resorted to reviewing all potentially problematic specimen types, such as brain biopsies, breast biopsies, pigmented skin lesions, and thyroid fine-needle aspirations or thyroid biopsies. The efficacy of focused reviews is just beginning to be investigated.22,23
Among our study's participants, the reason given for review of cases was the difficulty of diagnosis in 46% of cases and a required audit per departmental policy in 43% of cases. It is reassuring that nearly one-half of the cases would have been reviewed even if a policy did not exist. The existence of a policy, however, appears to prompt pathologists to have their cases reviewed by another pathologist: departments that had a policy for review of cases had higher review rates (9.6% versus 6.5%) and a higher percentage of the cases that they reviewed were malignant (48% versus 36%). So the observation made above, about review for malignancy of likely positives versus potentially false-negative cases holds here as well. The levels of review measured in this study seem to be workable. Targeted reviews with subspecialty knowledge appear to be beneficial, workable, and effective.23,24 A recent publication by Raab et al23 suggests that a focused review of cases detects errors at a higher level than a random review. If true, then the selection of cases to be reviewed may be at least as important as the rate of cases reviewed before sign-out.
In conclusion, routine case reviews before sign-out occurs at a median rate of 8.2%. The optimal level and case mix for review before sign-out remains unknown and deserves investigation to determine the best strategies for analytic error prevention in surgical pathology.
The authors have no relevant financial interest in the products or companies described in this article.
From the Department of Pathology, Mayo Clinic Florida, Jacksonville, Illinois (Dr Nakhleh); the Department of Pathology, Phoenixville Hospital, Phoenixville, Pennsylvania (Dr Bekeris); the Department of Statistics, College of American Pathologists, Northfield, Illinois (M. Souers); the Department of Pathology, Henry Ford Health System, Detroit, Michigan (Dr Meier); and the Department of Pathology, St Joseph's Mercy Hospital, Ann Arbor, Michigan (Dr Tworek).