Context.—To avoid errors many surgical pathology services mandate review of a case by a second pathologist before reports are released (signed out).

Objective.—To study the extent and characteristics of such review.

Design.—Participants retrospectively examined up to 400 cases to identify a maximum of 30 cases reviewed by at least one additional pathologist before sign-out. For each case, participants documented the organ system, primary disease type, number of additional pathologists consulted, and the reason for case review. The main outcome measure was the fraction of surgical pathology cases that underwent second pathologist review before sign-out.

Results.—From 45 laboratories, examination of 18 032 surgical pathology cases yielded 1183 (6.6%) cases that had been reviewed before sign-out. The median laboratory reviewed 8.2% of cases. Three-fifths of reviews focused on 4 organ systems: gastrointestinal (20.5%), breast (16.0%), skin (12.7%), and female genital tract (10.0%). Malignant neoplasm far exceeded all other categories of disease in reviewed cases (45.3%). Cases were reviewed by one additional pathologist 78% of the time. Two dominant reasons for case review emerged: difficult diagnosis (46.2%) and audit required by departmental policy (43.0%). Most laboratories (71%) had departmental policies regarding review of cases. These laboratories reviewed cases about 33% more often than laboratories without policies (9.6% versus 6.5%).

Conclusions.—Review of selected surgical pathology cases before sign-out is widely accepted with 71% of participant laboratories following policies to this effect. About 1 case in 15 (6.6%) were reviewed with the median laboratory of participants reviewing about 1 in 12 (8.2%).

Early in training, pathologists are taught to seek the opinion of others when confronted with difficult diagnostic cases. Most pathologists develop particular interest and knowledge in specific organs and tend to seek the advice of others when presented with challenging cases outside this domain of interest and knowledge. Until recently, seeking advice on cases was a voluntary exercise. Recent attention to diagnostic errors in the medical literature and the public press has led to many pathology practices standardizing such case reviews as a plausible measure to prevent errors and to provide assurance of diagnostic accuracy.111 

This trend has often been adopted through policies that require second pathologist review of particular types of case. The content of the case review tends to reflect previous specific experience with misdiagnosis in each surgical pathology practice. We report here a Q-Probes study of practices in 45 surgical pathology laboratories that reviewed cases before they were released (review before sign-out). To our knowledge, this is the first study to measure the frequency and characteristics of case review before sign-out in multiple institutions.

This study was offered and conducted as a Q-Probes study, the basic mechanism of which has been previously described.12 Briefly, Q-Probes studies are subscription-based quality assurance studies that are conducted at a certain point in time and have the advantage of peer group comparison to assist in determining level of performance.

This study was offered and conducted in the fourth quarter of 2008. Participants examined accessioned cases retrospectively from August 1, 2008, until 30 surgical pathology cases were identified that had documentation of review by at least a second pathologist before sign-out or until 400 consecutively accessioned cases had been examined.

For each identified, reviewed case, participants documented the organ system and primary disease type involved in the diagnosis, the number of additional pathologists consulted, and the reason for case review. Cases reviewed after sign-out and cases for which the second reviewer was a resident or fellow were excluded from the study. Participant laboratories also provided demographic information describing their laboratory and the institutions at which they serve, as well as practice variables that the study authors hypothesized might be linked to surgical pathology case review.

When data distributions for case review rate were statistically analyzed, the review rate was slightly skewed toward higher review rates, so a rank-based analysis for the univariate tests and log-transformation for the multivariate regression analysis were used. One participant had a 0% review rate and was excluded from the statistical analysis.

Individual associations between the review rate and the demographic and practice variables were investigated using Kruskal-Wallis tests for discrete-valued, independent variables and regression analysis for continuous, independent variables. Variables with significant associations (P < .10) were then introduced into a forward-selection, multivariate regression model. Variables remaining after this procedure would be significantly associated at the P < .05 significance level.

Of the 45 participant surgical pathology laboratories, most (89%) were located in the United States, 3 (7%) were in Saudi Arabia, and 1 (2%) each were in Australia and Canada; 37.8% (17 of 45) considered their laboratories to be teaching hospitals, and 22.2% (10 of 45) maintained pathology residency programs. Most of the laboratories (82.2%; 37 of 45) had been inspected by the Laboratory Accreditation program of the College of American Pathologists within 2 years of the study and 34.1% of institutions (15 of 44) had been inspected by the Joint Commission within the previous 2 years. Table 1 shows institutional demographic information about institutional type, bed size, location, and governmental affiliation. Table 2 shows the distribution of surgical pathology accessioned cases and of full-time pathologists who evaluated cases. The median participant laboratory processed 10 422 cases in 2007 and had 4 surgical pathologists as staff. Table 3 shows the characteristics that we hypothesized might influence case reviews; 32 of 45 laboratories (71.1%) had a policy regarding review of cases before case sign-out. Although most laboratories (24 of 45; 53%) did not target any percentage for review, 7 laboratories (15.6%) targeted 10% of cases for review, and 1 laboratory (2%) targeted 95% of cases.

Table 1

Institution Demographics

Institution Demographics
Institution Demographics
Table 2

Distribution for the Number of Surgical Specimens Accessioned in 2007 and Number of Full-Time Equivalent (FTE) Pathologists Who Sign-Out Surgical Pathology Specimens

Distribution for the Number of Surgical Specimens Accessioned in 2007 and Number of Full-Time Equivalent (FTE) Pathologists Who Sign-Out Surgical Pathology Specimens
Distribution for the Number of Surgical Specimens Accessioned in 2007 and Number of Full-Time Equivalent (FTE) Pathologists Who Sign-Out Surgical Pathology Specimens
Table 3

Practice Characteristics That May Be Related to Surgical Pathology Case Reviews

Practice Characteristics That May Be Related to Surgical Pathology Case Reviews
Practice Characteristics That May Be Related to Surgical Pathology Case Reviews

In aggregate, 1183 cases (6.6%) were reviewed among 18 032 cases examined. The median rate of review was 8.2% with a skewed distribution shown in Table 4. The characteristics of the study sample, the number of cases reviewed per participating laboratory, and the number of accessioned cases examined to find the study sample of reviewed cases is shown in Table 5. Most participants reached the study target of 30 cases in less than 400 cases examined. Table 6 shows the organ systems involved in the cases reviewed before sign-out. The 4 most frequent categories, gastrointestinal, breast, skin, and female genital tract cases, accounted for about 60% of reviewed cases. Table 7 shows the primary disease type, number of pathologists consulted, and the reason for case review before sign-out. As expected, malignant neoplasms (45.3%) were the highest proportion of cases. When combined with premalignant (7.4%), low malignant/borderline neoplasm (2.8%), and malignant mimics (benign proliferation/hyperplasia/atypia) (15.7%), this accounted for nearly three-quarters of all cases.

Table 4

Distribution of the Percentage of Cases Reviewed Before Sign-Out

Distribution of the Percentage of Cases Reviewed Before Sign-Out
Distribution of the Percentage of Cases Reviewed Before Sign-Out
Table 5

Distributions for the Number of Cases Reviewed Before Sign-Out and the Total Number of Cases Examined During the Study Period

Distributions for the Number of Cases Reviewed Before Sign-Out and the Total Number of Cases Examined During the Study Period
Distributions for the Number of Cases Reviewed Before Sign-Out and the Total Number of Cases Examined During the Study Period
Table 6

The Frequency of Organ Systems Listed in Cases Reviewed Before Sign-Out

The Frequency of Organ Systems Listed in Cases Reviewed Before Sign-Out
The Frequency of Organ Systems Listed in Cases Reviewed Before Sign-Out
Table 7

The Frequency of Disease Type, Number of Additional Pathologists Consulted, and Reason for Review of Cases Reviewed Before Sign-Out

The Frequency of Disease Type, Number of Additional Pathologists Consulted, and Reason for Review of Cases Reviewed Before Sign-Out
The Frequency of Disease Type, Number of Additional Pathologists Consulted, and Reason for Review of Cases Reviewed Before Sign-Out

In evaluating practice and demographic variables, 4 factors were significantly associated with the level of review. Laboratory policy regarding case reviews before sign-out was associated with a higher percentage of reviewed cases (9.6% versus 6.5%). Nonfederal governmental institutions had a significantly lower rate of review versus nongovernmental and federal governmental facilities (3.5% versus 8.4% and 12.4%, respectively). Laboratories with review policies also reviewed malignancies at a higher frequency than laboratories without such policies (P < .001; 48% versus 36% of reviewed cases). Laboratories with review policies also tended to review breast cases at a higher rate than those that did not (P  =  .003; median rate of 1.2% versus 0.5% of all cases). No such statistical significance was found with other organ systems.

To our knowledge, this is the first multi-institutional study to measure the extent and characteristics of pathology case review before case sign-out. The median laboratory reviewed 8.2% of cases with an aggregate rate of 6.6%. One other publication has attempted to measure the rate of case review before sign-out. Renshaw and Gould13 measured their institutional rate at 13%.

In our study, the range of the rate of case review was wide, laboratories in the 10th and 90th percentiles reviewed 2.0% and 17.1% of cases. This, however, likely represents a low estimate of the rate of intradepartmental consultation. Because this study only captures documented reviews, we can speculate that a number of cases are routinely shown to other pathologists but are not documented and, therefore, are not captured by a retrospective case review. In a previous Q-Probes study14 of expert extradepartmental consultation, a median rate of 0.5% was recorded with a range of 0% to 2.0%. In that study, expert extradepartmental consultations only represented consultations that were initiated by the pathologists and did not include cases that are seen at other institutions after sign-out because the patient seeks another opinion or goes to another institution for definitive treatment. The rate described in this report represents the prevalent rate of documented cases reviewed internally before sign-out.

Several quality-assurance guidelines have proposed a 10% review rate as a standard, without demonstrating the effectiveness of this specific rate of review.15,16 These mandated quality-assurance reviews are typically performed after cases are signed out and are used to evaluate error rates of an individual or a program.17 So they differ in timing—after sign-out versus before sign-out—and in intention—error detection versus error prevention—from the double-review practice examined in our study. Several studies have hypothesized that review of cases before sign-out reduces error and amended report rates.13,18,19 Our study did not collect error or amended report rates, so it cannot speak to that hypothesis. If potential errors can be identified before reports are issued, this preventative measure would indeed be an important quality measure. It is surprising to the authors of this study that the effectiveness of this procedure has not been tested. The appropriate rate and pattern of such reviews has not been empirically determined. Our study at least offers a picture of the extent of the practice at present.

Regarding patterns of review, we determined what types of cases were reviewed by study participants. The top 4 organ systems reviewed were gastrointestinal tract (20%), breast (16%), skin (13%), and the female genital tract (10%). Although the proportion of cases seen in a particular laboratory varies depending on the practice setting, GI, skin, and female genital tract specimens tend to be among the highest proportion of cases in most North American practice settings. In general practice, breast specimens tend to account for a small proportion of cases but are probably reviewed at a much higher level than other types of specimens because of the clinical implication of breast biopsy results. This is borne out in our statistical analysis where breast cases were reviewed at a higher median rate (1.2%) in laboratories with a review policy than in laboratories that did not have such a policy (0.5%). With respect to disease type, malignancy accounted for 45% of all reviewed cases and far exceeded any other category by a margin of 3 to 1. Selection of known malignancies for review is easy, but this review criterion leaves aside the issue of detecting cases of neoplasms that potentially could have been missed. These may have been misplaced in the second-most frequent category of benign neoplasm/hyperplasia/atypia. False-negative cases indeed represent the highest proportion (73% and 61%) of cases in 2 reports of legal verdict and settlements against pathologists.20,21 Some laboratories have resorted to reviewing all potentially problematic specimen types, such as brain biopsies, breast biopsies, pigmented skin lesions, and thyroid fine-needle aspirations or thyroid biopsies. The efficacy of focused reviews is just beginning to be investigated.22,23 

Among our study's participants, the reason given for review of cases was the difficulty of diagnosis in 46% of cases and a required audit per departmental policy in 43% of cases. It is reassuring that nearly one-half of the cases would have been reviewed even if a policy did not exist. The existence of a policy, however, appears to prompt pathologists to have their cases reviewed by another pathologist: departments that had a policy for review of cases had higher review rates (9.6% versus 6.5%) and a higher percentage of the cases that they reviewed were malignant (48% versus 36%). So the observation made above, about review for malignancy of likely positives versus potentially false-negative cases holds here as well. The levels of review measured in this study seem to be workable. Targeted reviews with subspecialty knowledge appear to be beneficial, workable, and effective.23,24 A recent publication by Raab et al23 suggests that a focused review of cases detects errors at a higher level than a random review. If true, then the selection of cases to be reviewed may be at least as important as the rate of cases reviewed before sign-out.

In conclusion, routine case reviews before sign-out occurs at a median rate of 8.2%. The optimal level and case mix for review before sign-out remains unknown and deserves investigation to determine the best strategies for analytic error prevention in surgical pathology.

1
Kohn
,
L. T.
,
J. M.
Corrigan
, and
M. S.
Donaldson
.
eds.
To Err Is Human: Building a Safer Health System
.
Washington, DC
National Academy Press
.
1999
.
2
Sirota
,
R. L.
The Institute of Medicine's report on medical errors: implications for pathology.
Arch Pathol Lab Med
2000
.
124
(
11
):
1674
1678
.
3
Fisher
,
B.
U of C Hospital sued for error that resulted in removal of breast.
Chicago Sun Times
5
11
2005
.
4
Goodman
,
E.
When doctors make mistakes.
Good Housekeeping
4
2001
.
5
Parker-Pope
,
T.
Risk of error may justify second opinion on pathology reports.
Wall Street Journal
4
13
2001
.
1
.
6
Parker-Pope
,
T.
Why it's hard to get a second opinion (and how to make sure you get one).
Wall Street Journal
2
1
2005
.
1
.
7
Renshaw
,
A. A.
,
N. E.
Pinnar
,
M. R.
Jiroutek
, and
M. L.
Young
.
Blinded review as a method for quality improvement in surgical pathology.
Arch Pathol Lab Med
2002
.
126
(
8
):
961
963
.
8
Westra
,
W. H.
,
J. D.
Kronz
, and
D. W.
Eisele
.
The impact of second opinion surgical pathology on the practice of head and neck surgery: a decade experience at a large referral hospital.
Head Neck
2002
.
24
(
7
):
684
693
.
9
Hahm
,
G. K.
,
T. H.
Niemann
,
J. G.
Luca
, and
W. L.
Frankel
.
The value of second opinion in gastrointestinal and liver pathology.
Arch Pathol Lab Med
2001
.
125
(
6
):
736
739
.
10
Santoso
,
J. T.
,
R. L.
Coleman
,
R. L.
Voet
,
S. G.
Bernstein
,
S.
Lifshitz
, and
D.
Miller
.
Pathology slide review in gynecologic oncology.
Obstet Gynecol
1998
.
91
(
5, pt 1
):
730
734
.
11
Kronz
,
J. D.
,
W. H.
Westra
, and
J.
Epstein
.
Mandatory second opinion surgical pathology at a large referral hospital.
Cancer
1999
.
86
(
11
):
2426
2435
.
12
Bachner
,
P.
and
P.
Howanitz
.
Using Q-Probes to improve the quality of laboratory medicine: a quality improvement program of the College of American Pathologists.
Qual Assur Health Care
1991
.
3
(
3
):
167
177
.
13
Renshaw
,
A. A.
and
E. W.
Gould
.
Measuring the value of review of pathology material by a second pathologist.
Am J Clin Pathol
2006
.
125
(
5
):
737
739
.
14
Azam
,
M.
and
R. E.
Nakhleh
.
Surgical pathology extradepartmental consultation practices: a College of American Pathologists' Q-Probes study of 2428 consultations from 180 laboratories.
Arch Pathol Lab Med
2002
.
126
(
4
):
405
412
.
15
Clinical Laboratory Improvement Amendments of 1988; final rule (42 CFR Part 405, et al)
Fed Regist
1992
.
57
:
7001
7186
.
16
Association of Directors of Anatomic and Surgical Pathology
Recommendations on quality control and quality assurance in anatomic pathology.
Am J Surg Pathol
1991
.
15
(
10
):
1007
1009
.
17
Nakhleh
,
R. E.
and
P. L.
Fitzgibbons
.
eds.
Quality Management in Anatomic Pathology: Promoting Patient Safety Through Systems Improvement and Error Reduction
.
Northfield, IL
College of American Pathologists
.
2005
.
18
Nakhleh
,
R. E.
and
R. J.
Zarbo
.
Amended reports in surgical pathology and implications for diagnostic error detection and avoidance: a College of American Pathologists' Q-Probes study of 1 667 547 accessioned cases in 359 laboratories.
Arch Pathol Lab Med
1998
.
22
(
4
):
303
309
.
19
Novis
,
D.
Routine review of surgical pathology cases as a method by which to reduce diagnostic errors in a community hospital.
Pathol Case Rev
2005
.
10
(
2
):
63
67
.
20
Kornstein
,
M. J.
and
S. P.
Byrne
.
The medicolegal aspect of error in pathology: a search of jury verdicts and settlements.
Arch Pathol Lab Med
2007
.
131
(
4
):
615
618
.
21
Troxel
,
D. B.
Medicolegal aspects of error in pathology.
Arch Pathol Lab Med
2006
.
130
(
5
):
617
619
.
22
Zarbo
,
R. J.
,
F. A.
Meier
, and
S. S.
Raab
.
Error detection in anatomic pathology.
Arch Pathol Lab Med
2005
.
129
(
10
):
1237
1245
.
23
Raab
,
S. S.
,
D. M.
Grysbicki
,
L. K.
Mahood
,
A. V.
Parwani
,
S. F.
Kwan
, and
U. N.
Rao
.
Effectiveness of random and focused review in detecting surgical pathology error.
Am J Clin Pathol
2008
.
130
(
6
):
905
912
.
24
Meier
,
F. A.
,
R. C.
Varney
,
R.
D'angelo
,
A.
Gandhi
, and
R. J.
Zarbo
.
A taxonomy of amended reports assesses impact of Henry Ford Production System: a lean quality initiative in surgical pathology.
Mod Pathol
2009
.
22
(
suppl 1
):
365A
.

The authors have no relevant financial interest in the products or companies described in this article.

Author notes

From the Department of Pathology, Mayo Clinic Florida, Jacksonville, Illinois (Dr Nakhleh); the Department of Pathology, Phoenixville Hospital, Phoenixville, Pennsylvania (Dr Bekeris); the Department of Statistics, College of American Pathologists, Northfield, Illinois (M. Souers); the Department of Pathology, Henry Ford Health System, Detroit, Michigan (Dr Meier); and the Department of Pathology, St Joseph's Mercy Hospital, Ann Arbor, Michigan (Dr Tworek).