Additional reviews of diagnostic surgical and cytology cases have been shown to detect diagnostic discrepancies.
To develop, through a systematic review of the literature, recommendations for the review of pathology cases to detect or prevent interpretive diagnostic errors.
The College of American Pathologists Pathology and Laboratory Quality Center in association with the Association of Directors of Anatomic and Surgical Pathology convened an expert panel to develop an evidence-based guideline to help define the role of case reviews in surgical pathology and cytology. A literature search was conducted to gather data on the review of cases in surgical pathology and cytology.
The panel drafted 5 recommendations, with strong agreement from open comment period participants ranging from 87% to 93%. The recommendations are: (1) anatomic pathologists should develop procedures for the review of selected pathology cases to detect disagreements and potential interpretive errors; (2) anatomic pathologists should perform case reviews in a timely manner to avoid having a negative impact on patient care; (3) anatomic pathologists should have documented case review procedures that are relevant to their practice setting; (4) anatomic pathologists should continuously monitor and document the results of case reviews; and (5) if pathology case reviews show poor agreement within a defined case type, anatomic pathologists should take steps to improve agreement.
Evidence exists that case reviews detect errors; therefore, the expert panel recommends that anatomic pathologists develop procedures for the review of pathology cases to detect disagreements and potential interpretive errors, in order to improve the quality of patient care.
The test cycle in surgical pathology and cytology is similar to the test cycle of other laboratory tests.1–4 It is composed of the preanalytic, analytic, and postanalytic phases. The preanalytic and postanalytic phases are very similar to tests in the clinical laboratory. The preanalytic phase is composed of specimen acquisition, specimen labeling, and delivery to the laboratory, where the specimen is prepared for the analytic phase. The postanalytic phase begins with report generation and ends with delivery of the report to the clinician.
Unlike the other phases of the test cycle, the analytic phase is substantially different in surgical pathology and cytology (versus clinical pathology) in that it involves the inherent judgment of the pathologist at the time of slide interpretation.5–9 It is therefore more subjective than clinical laboratory tests. There are many factors that contribute to an accurate interpretive diagnosis, including: (1) the pathologist's knowledge and experience, (2) clinical correlation, (3) standardized diagnostic criteria and taxonomy, (4) confirmatory ancillary studies when available, and (5) secondary review of cases.
Studies have shown the additive value of clinical correlation, standardization of diagnostic criteria, and taxonomy and confirmatory ancillary testing to the accuracy of surgical pathology and cytology diagnoses.5,10–15 Several of these factors contribute to establishing a precise diagnosis, but the pathologist's knowledge and experience remain the essential factors in interpretive diagnosis. Although numerous studies have shown that case reviews help detect interpretive diagnostic errors, there have been no efforts to formalize this practice as a strategy to reduce errors. In considering processes occurring in surgical pathology and cytology, targeted case reviews could be an integral component of a quality assurance plan that is aimed proactively at preventing errors before they have a potential adverse impact on patient care.
The College of American Pathologists (CAP) Pathology and Laboratory Quality Center in association with the Association of Directors of Anatomic and Surgical Pathology (ADASP) convened an expert panel to systematically review the literature and develop an evidence-based guideline to help define the role of case reviews in surgical pathology and cytology. We focused on the contribution of case reviews to error detection and prevention of interpretive diagnosis.
The CAP Pathology and Laboratory Quality Center (the Center) and ADASP convened an expert panel consisting of practicing pathologists with expertise and experience in surgical pathology. Members included practicing pathologists in the United States and Canada. The CAP and ADASP approved the appointment of the project, cochairs, and expert panel members. In addition, a physician-methodologist experienced in systematic review and guideline development consulted with the panel throughout the project.
CONFLICT OF INTEREST POLICY
Prior to acceptance on the expert panel, potential members completed the CAP conflict of interest disclosure process, whose policy and form require disclosure of material financial interest in, or potential for benefit of significant value from, the guideline's development or its recommendations. The potential members completed the conflict of interest disclosure form, listing any relationship that could be interpreted as constituting an actual, potential, or apparent conflict. Potential conflicts were managed by the cochairs. Everyone was required to disclose conflicts prior to beginning and continuously throughout the project's timeline. Disclosed conflicts of the expert panel members are listed in the Appendix. The CAP and ADASP provided funding for the administration of the project; no industry funds were used in the development of the guideline. All panel members volunteered their time and were not compensated for their involvement. Please see the supplemental digital content (SDC) available at www.archivesofpathology.org in the January 2016 table of contents for full details on the conflict of interest policy.
The panel addressed the overarching question, “What are the most effective ways to reduce interpretive diagnostic errors in Anatomic Pathology?” The key questions that the panel addressed were:
Does targeted review (done at either the analytic or the postanalytic phase) of surgical pathology or cytology cases (slides and/or reports) reduce the error rate (often measured as amended reports) or increase the rate of interpretive error detection compared with no review, random review, or usual review procedures?
What methods of selecting cases for review have been shown to increase/decrease the rate of interpretive error detection compared with no review, random review, or usual review procedures?
A detailed account of the methods used to create this guideline can be found in the SDC, including additional scope questions.
Systematic Literature Review and Analysis
A systematic literature search was completed for relevant evidence in MEDLINE using both OvidSP and PubMed (January 1, 1992, to October 31, 2012). The search strategy included medical subject headings and text words to capture the general concepts of pathology and quality (eg, pathology, surgical; pathology, clinical; pathology and quality improvement; quality assurance, health care; quality control; reproducibility of results), and a targeted concept of slide/case review. MEDLINE searches were supplemented with a search of Google Scholar, a search for meeting abstracts (2008–2012) using both Biosis Previews and hand searching, and a focused hand search of identified pathology journals (2008–2012). An update of the OvidSP search was conducted through October 2013. All searches were limited to human studies published in English. Reference lists of included articles were also reviewed for relevant reports. Detailed information regarding the literature search strategy can be found in the SDC.
Eligible Study Designs
All study designs were included in the initial literature search. In addition to journal articles, the search identified monographs and meeting abstracts. During evidence review, articles that did not present new evidence were excluded, including letters, commentaries, and editorials.
Published studies were selected for full-text review if they met each of the following criteria:
English-language articles/documents that addressed surgical pathology or cytology studies and provided data or information relevant to one or more key questions; and
Original research addressing pathology case reviews.
Editorials, letters, commentaries, and invited opinions were not included in the study. Articles were also excluded if the full article was not available in English, did not address any key questions, and/or focused primarily on clinical pathology studies, including all other specialties except radiology. Articles were also excluded if they were focused on any of the following: preanalytic specimen processes, noninterpretative postanalytic processes, additional diagnostic techniques, issues related to competency use of checklists, standardized language, taxonomy, or formatting.
Outcomes of Interest
The panel assessed studies that identified discrepancies in interpretation between a primary pathologist review and a second pathologist review as a way of estimating the error rate. To the extent that erroneous readings can be identified in excess of an expected degree of disagreement, then a method of targeted review would be said to be effective. Thus, studies with a control group are desirable; as a practical matter, however, it is necessary to examine uncontrolled series, too. Studies had to report numbers of discrepant diagnoses among a defined population of specimens examined in order to allow calculation of a discrepancy rate.
The panel did not assess discrepancies from the preanalytic specimen process (related to tissue collection and processing) or postanalytic errors (eg, typographic or transcription errors, amended reports), additional diagnostic techniques (eg, immunomarkers), issues related to competency, or the use of checklists, standardized language, taxonomy, or formatting.
Various studies classify errors in different ways (eg, major versus minor, clinically significant versus insignificant). Recognizing that all errors are not alike, we assessed the severity of interpretive errors according to the clinical impact on a patient.16 We considered the clinical impact of errors as follows: (1) diagnostic thinking (error results in a change in diagnosis or diagnostic category); (2) therapeutic efficacy (error results in a change in therapeutic choice); or (3) patient outcome efficacy (error results in a change in outcome [eg, procedure avoided]; demonstrating this unequivocally may require long-term follow-up). We also considered the efficiency or cost (in terms of effort or dollars) that a targeted review strategy entails.
The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach provides a system for rating quality of evidence and strength of recommendations that is explicit, comprehensive, transparent, and pragmatic, and is increasingly being adopted by organizations worldwide.17 The GRADE approach examines the quality of evidence at the level of individual studies and also at the review level. GRADE was used for rating the quality of evidence. At the individual study level, we assessed studies according to three criteria: (1) study design rating, (2) risk of bias rating, and (3) applicability concerns. For a full description of the assessment, refer to the SDC. A physician-methodologist consultant experienced in systematic review and guideline development rated the quality of each study, constructed evidence tables and summary of findings tables, and, along with the panel, developed quality of evidence ratings. The quality of evidence definitions from GRADE are shown in Table 1.
Assessing the Strength of Recommendations
Development of recommendations requires that the panel review the identified evidence and make a series of key judgments (using procedures described in the SDC). CAP uses a three-tier system to rate the strength of recommendations instead of the traditional two-tier approach of strong or weak recommendations. This approach is consistent with prior CAP guidelines (Table 2).
This guideline will be reviewed every 4 years, or earlier in the event of publication of substantive and high-quality evidence that could potentially alter the original guideline recommendations. If necessary, the entire panel will reconvene to discuss potential changes. When appropriate, the panel will recommend revision of the guideline to CAP and ADASP for review and approval.
The CAP developed the Pathology and Laboratory Quality Center as a forum to create and maintain evidence-based practice guidelines and consensus statements. Practice guidelines and consensus statements reflect the best available evidence and expert consensus supported in practice. They are intended to assist physicians and patients in clinical decision-making and to identify questions and settings for further research. With the rapid flow of scientific information, new evidence may emerge between the time a practice guideline or consensus statement is developed and when it is published or read. Guidelines and statements are not continually updated and may not reflect the most recent evidence. Guidelines and statements address only the topics specifically identified therein and are not applicable to other interventions, diseases, or stages of diseases. Furthermore, guidelines and consensus statements cannot account for individual variation among patients and cannot be considered inclusive of all proper methods of care or exclusive of other treatments. It is the responsibility of the treating physician or other health care provider, relying on independent experience and knowledge, to determine the best course of treatment for the patient. Accordingly, adherence to any practice guideline or consensus statement is voluntary, with the ultimate determination regarding its application to be made by the physician in light of each patient's individual circumstances and preferences. CAP and ADASP make no warranty, express or implied, regarding guidelines and statements, and specifically exclude any warranties of merchantability and fitness for a particular use or purpose. CAP and ADASP assume no responsibility for any injury or damage to persons or property arising out of or related to any use of this statement or for any errors or omissions.
A total of 823 studies met the search term requirements. A total of 137 articles were included for data extraction. Excluded articles were available as discussion or background references. The panel convened 26 times (25 by teleconference and 1 face-to-face meeting) to develop the scope, draft recommendations, review and respond to solicited feedback, and assess the quality of evidence that supports the final recommendations. A nominal group technique was employed by the panel for consensus decision-making to encourage unique input with balanced participation among group members. An open comment period was held from December 2, 2013, through January 21, 2014, during which draft recommendations were posted on the CAP Web site. Five recommendations were drafted, with strong agreement for each recommendation from the open comment period participants ranging from 87% to 93% (refer to Outcomes in the SDC for full details). The expert panel modified the draft recommendations based on the feedback given during the considered judgment process. Then, an independent review panel, masked to the expert panel and vetted through the conflict of interest process, provided final review of the manuscript and recommended it for approval by the CAP.
All articles discussed the review of cases by one or more additional pathologist(s). Table 3 shows a summary of the studies that demonstrated a discrepancy rate. The studies could be grouped by service (surgical pathology versus cytology), organ system, and type of review (internal versus external). Most studies expressed a discrepancy rate. A smaller percentage conveyed a major or significant discrepancy rate. In our review 81 different definitions of major or significant diagnostic discrepancies were used. The entries in this array of definitions sort themselves into three general categories that characterize major or significant discrepancies: (1) differences with demonstrated impact on patient care, (2) differences with potential impact on patient care, and (3) differences that indicate substantial diagnostic changes (ie, benign and malignant or positive and negative diagnoses), without regard to actual or potential clinical impact. The final recommendations are summarized in Table 4.
—Anatomic pathologists should develop procedures for the review of pathology cases to detect disagreements and potential interpretive errors, and to improve patient care.
The literature review demonstrated that a review of cases detects discrepancies and errors, and furthermore, that discrepancy or error rates fall in a range that is clinically important. The evidence was inadequate to demonstrate a direct impact on patient safety, because few studies reported the clinical impact on patient outcomes that resulted from interpretive errors. The overall GRADE quality of evidence was low, but because of the consistent findings of a large number of studies of clinically important major discrepancy rates, and the significant impact that a diagnostic error may be expected to have on an affected individual, the panel graded this guideline statement as a CAP “recommendation.” Because considerable variability of methods and rates exists, the evidence was lacking to determine a precise prevalent rate of discrepancy or error.
Numerous studies have demonstrated that the review of cases by a second pathologist detects disagreements and potential errors. Significant error rates range from 0.1% to more than 10%, depending on the method of review and the type of cases. The manner and extent of case reviews are outlined in the following recommendations, but they should be tailored to the needs of the individual laboratory. Ideally, the goal is to enhance collaborative diagnostic teamwork and reduce errors.
Data extraction was performed on 137 articles that described some type of surgical pathology and/or cytology case review. A subset of those articles is summarized in Table 3. All articles that underwent data extraction determined that review of cases found errors or discrepancies in diagnosis. Some articles defined errors as “significant” or “major” (see definition in the “Outcome” section above). Although this was a smaller proportion of cases, every study determined that these discrepancies were detected by review of cases, leading to the conclusion that case reviews have the potential to reduce errors and improve patient care. One program, Cancer Care Ontario, has undertaken a similar analysis of the literature and recommends selective review of oncologic cases.18
—Anatomic pathologists should perform case reviews in a timely manner to have a positive impact on patient care.
The literature review found 4 moderate-quality comparative studies that show prospective reviews (before sign-out) compared with retrospective reviews (after sign-out) can reduce disagreement/major disagreement rates and amended report rates (Table 4). The evidence was inadequate to demonstrate a direct impact on patient safety because few studies reported patient outcomes that resulted from interpretive errors. The GRADE quality of evidence is low, but because of the consistent findings in these 4 studies and no contradictory studies, the panel graded this guideline statement as a CAP “recommendation.”
Secondary reviews of surgical pathology and cytology cases should be performed in a timely manner to ensure appropriate treatment decisions and patient care. Ideally, prospective reviews are encouraged if possible, and resources should be directed to the timely review of cases prior to rendering diagnoses. Nevertheless, in some circumstances retrospective reviews may also be useful—such as when prospective reviews are not possible because of various laboratory limitations and constraints usually related to turnaround time requirements—but it is expected that these reviews also occur in a timely manner. In addition, there are specific settings—for example, clinical correlation conferences and correlating cytology/biopsy cases with excision specimens—where retrospective secondary reviews may be a standard process. The preference for prospective review stated above should not be interpreted as a justification for no longer performing these retrospective reviews.
If review of cases is to have an impact on patient care, then detection of errors must occur prior to definitive treatment. Detection of errors at this point also prevents unnecessary and potentially harmful therapies. The ideal time for review of cases is before cases are signed out. This is best for several reasons: (1) the correct treatment plan may be constructed based on the reviewed diagnosis, (2) this minimizes any rework or amendment of reports, (3) this minimizes any potential confusion about the diagnosis, and (4) this builds confidence and trust in the laboratory system. Some articles demonstrated that review of cases before sign-out is associated with a decrease in amended reports within the same laboratory, particularly amended reports for diagnostic edits (Table 5).19–22 An additional article demonstrates an association of lower amended report rates in laboratories that perform prospective reviews versus laboratories that perform retrospective reviews.23
The definition of timely can be significantly affected by communication with the clinician. For example, when cases are sent out of the laboratory for second opinion, these second opinions may take days or even weeks to come back. If the clinician is notified and is deferring treatment until the diagnosis becomes available, the consultation is still considered timely. In a similar manner, in any case where the second in-house review may take any significant length of time to perform, clinician notification of this process would still be considered a timely review.
On the other hand, review of cases after sign-out is well established and accepted. Examples include review of cases for many types of multidisciplinary conferences. These reviews tend to be incorporated into the normal workflow for case management and with the treatment decisions frequently deferred to and based on the multidisciplinary discussions at these conferences. Although these reviews may lead to amended reports and some rework, they tend to be timely and lead to appropriate patient care.
Review of cases temporally distant from when the case was originally signed out has the potential to be devastating for the patient as well as the pathologist. Identifying an error months or years after the case is finalized may not help the patient, since it is likely the patient has already been managed based on the original diagnosis. Alternatively, it may lead to the realization of a diagnosis not previously known, possibly resulting in delay of appropriate therapy with the potential for a more advanced stage of disease.
3. Expert Consensus Opinion
—Anatomic pathologists should have documented case review procedures that are relevant to their practice setting.
The quality of evidence was low to support using case review procedures compared with no case review procedures and to support targeted reviews versus random case review procedures; however, the evidence was very low with regard to distinction between different methods of review. The overall GRADE quality of evidence was very low, leading the panel to rate this guideline statement with the CAP strength of recommendation of “expert consensus opinion.”
A wide variety of review processes have been reported in the literature, with varying levels of error detection of interpretive diagnoses. Depending on the method, reviews may also affect turnaround time, increase workload, and add expense. The ideal method may depend on the practice setting, which may range from a small general community practice to a highly subspecialized academic practice. Therefore, the review processes should be tailored to the type of practice setting in order to maximize error detection while minimizing negative impacts. Targeted review, general review, percentage of cases reviewed, blinded review, review of cases with known high rates of missed lesions, and other methods should be considered to design a system best suited for individual practices. The laboratory medical director is responsible for determining the policy.
Departments of pathology should formalize their review processes and include any and all review procedures in their quality assurance plan for surgical pathology and cytology. A plan to conduct and document review of cases may be the last opportunity to identify a diagnostic error. Although many types of reviews have been reported, departments should routinely document when departmental reviews occur. These reviews may include but are not limited to: (1) review of a selected percentage of cases, (2) review of selected types of diagnoses, (3) review of a selected organ system or specimen type, (4) review of random cases, (5) review of cases for multidisciplinary conferences, (6) in-house cases sent outside for review, (7) review of cases during cytology-histology correlation, (8) review of cases in a consensus conference, and (9) review of cases for any other reason.
Important principles to consider when reviewing cases include: (1) The reviewing pathologist should independently formulate opinions without influence from others. Some have suggested that blinded reviews are optimal, but this has not clearly been demonstrated to be necessary.24–26 (2) The reviewing pathologist ideally should have sufficient expertise in the material he or she is reviewing. (3) Case reviews performed prior to sign-out could be used to build collaborative teamwork and are excellent opportunities for pathologists to learn and improve their skills. (4) Review strategies should include negative cases because many errors are false negatives.27,28 (5) Targeted review of selected organs or diseases leads to detection of more errors compared with review of cases randomly.29
When review of cases occurs, the actual review should be documented in some fashion. When cases are reviewed before sign-out, many departments choose to document the review within the body of the report. Reviews may also be documented in a separate quality assurance log, either on paper or electronically. Another option is to have reviews be documented as addenda to the report. Case logs of multidisciplinary conferences or other types of conferences may also be included with the quality assurance documentation of reviews.
Defining adequate review policies may be difficult in specific settings. First, one of the most problematic settings to define adequate review policies is that of the solo pathologist, where there is no other pathologist in-house with whom to share the case. Essentially all of the published data involve a second pathologist reviewing the work of the initial pathologist. Although there may be value in the practice, there are few or no data available concerning a single pathologist reviewing his or her own work a second time. At a minimum, an adequate policy in this setting might consist of documenting review of cases for clinical conferences and all cases that are reviewed at an outside laboratory.
Second, for small pathology groups with more than one pathologist, although in-house review of cases by a second pathologist is encouraged, in this setting a review strategy similar to that for a solo practitioner may also suffice. Third, in some tertiary care settings, very few cases may be sent to an outside laboratory for a second opinion. However, many, if not all, of these tertiary settings have large subspecialized pathology groups. In this setting, documentation of cases reviewed at clinical conferences and cases that are shown within and between these subspecialty groups might qualify as the minimum review strategy.
4. Expert Consensus Opinion
—Anatomic pathologists should continuously monitor and document the results of case review.
The quality of evidence based on agreement studies was low for the finding that for several defined diagnoses and/or organ systems interobserver agreement is poor. In the panel's literature review there were no studies that directly related continuous monitoring to diagnostic agreement or improvement. The GRADE quality of evidence was very low, leading the panel to rate this guideline statement with the CAP strength of recommendation of “expert consensus opinion.” However, based on traditional quality assurance principles it is our expert consensus opinion that monitoring is necessary to maintain confidence in the quality of diagnoses. Once the review design is established, the review procedure should be adhered to and monitored, ensuring that the program is functioning as intended and that all anatomic pathologists are compliant. This may be assessed by such methods as monitoring overall rates of case review pre–sign-out or post–sign-out, monitoring amended/revised report rates, monitoring minor/major discrepancies, leveraging laboratory information system software to include review status in the permanent electronic records of cases, and/or making the review process and results part of ongoing professional practice evaluation.
In addition, the results of the reviews should be assessed for areas to investigate and evaluate additional questions about the practice, such as elucidating previously unidentified areas where local variability exists, identifying specific case types where additional internal or external expertise might be of value, recognizing individual practitioners who may need to acquire additional proficiency, and detecting systems and process issues that contribute to diagnostic disagreements and potential errors.
In monitoring and documenting the results of the review process, individual practices should operate with the knowledge that some diagnoses have a known reputation for high interobserver variability, and that for such cases “disagreements” between observers may not indicate an interpretive error, per se. Examples of such diagnoses include, but are not limited to, epithelial dysplasia in inflammatory bowel disease and in the setting of Barrett metaplasia of the esophagus, reactive versus squamous intraepithelial lesions in the uterine cervix, and Gleason grading of prostatic adenocarcinoma (Table 6).
Laboratories should develop written procedures and record the results of their intradepartmental review studies. A simple method to document the department's effort in review is to measure the percentage (or rate) of cases reviewed by a second pathologist prior to sign-out. A recent study has documented that departments with review policies reexamine cases at a rate of about 10%.30 The total percentage of cases that are reviewed before and after sign-out is not known.
If a department implements a policy of reviewing specific types of cases, then periodic audits of that case type may be indicated. Others commonly track amended reports with diagnostic edits by the sign-out pathologist or cases that have undergone external diagnostic review after sign-out; the number and percentage of cases with minor or major diagnostic agreements may be tabulated for the department and for individual practitioners. In some cases, it may not be entirely clear to pathologists whether a difference in opinion should result in an amendment or addendum to the report. Should this approach be chosen, the distinction between amendments and addenda used by the practice must be explicit, and participating pathologists must adhere to the distinction consistently. Additional methods to assess diagnostic proficiency may include frozen section–paraffin section correlation and cytology–follow-up tissue biopsy correlation; these may also be incorporated into an overall quality improvement program.
Other methods of documenting diagnostic proficiency are also acceptable.
5. Expert Consensus Opinion
—If pathology case reviews show poor agreement within a defined area, anatomic pathologists should take steps to improve agreement.
The quality of evidence was low regarding the best methods to improve agreement in areas for which agreement is poor. It is likely that best approaches may differ based on features of disease, individual practice patterns, and available ancillary diagnostic tests. Studies on methods to improve agreement were not found in our literature search or were outside the scope of our search. Therefore, the GRADE quality of evidence was not assessed, leading the panel to rate this guideline statement with the CAP strength of recommendation of “expert consensus opinion.”
The causes for poor agreement within and among anatomic pathology groups are variable. Two external factors need to be taken into account when assessing interobserver disagreement. First, some diagnoses have inherently higher interobserver variation than others, and these differences should be acknowledged (Table 6). Second, pathology diagnoses are dynamic and terminology changes; different designations for the same entity should be recognized and remedied as a problem distinct from that of interobserver variation itself. If poor interobserver agreement is discovered within a practice, particularly a degree of disagreement that exceeds that in published norms, then the practice members should use specific improvement methods (eg, consensus conferences, calibration slide sets, etc) to improve team consensus.
The quality of evidence was very low regarding the best methods to improve agreement in areas for which agreement is poor. It is likely that best approaches may differ based on features of disease, individual practice patterns, and available ancillary diagnostic tests.
Pathologists' experiences, expertise, and abilities are variable regarding some lesions with demonstrated high rates of diagnostic discrepancies. Examples include the assessment of thyroid lesions by fine-needle aspiration cytology and the assessment of esophageal dysplasia in the setting of Barrett esophagus. The use of standardized diagnostic criteria is a powerful approach to improve diagnostic variation.31–33 Methods that have been shown to reduce interobserver disagreement include intradepartmental consensus conferences with the acceptance and use of uniform diagnostic criteria. The use of calibration slide sets has also been used. Reinforcement of agreed-upon criteria works well with review and discussion of problematic cases. Disagreement rates due to other factors should also be addressed. These may include but are not limited to lack of familiarity with certain conditions and recent changes in diagnostic criteria or procedures.
Because secondary review of cases detects and corrects errors, it is natural to wonder whether these data can be used to measure quality in an anatomic pathology laboratory. Such a measure would be of tremendous interest to pathologists, clinicians, employers, insurers, and patients. However, at present it is not clear how best to interpret the results of these reviews appropriately, and these results should not be used to attempt to compare the quality of two different pathology laboratories. The reasons for this include but are not limited to:
The source of errors in different laboratories may differ.
The definition of error in different laboratories may differ.
The clinical significance of errors may differ.
The methods used to detect errors may differ.
The sensitivity of the review method is not controlled for and is unknown.
Expected ranges of performance are not well defined.
At present, there is supporting evidence to state that:
Second reviews successfully detect and reduce errors.
Groups that do second reviews have a lower error rate than if they did not perform second reviews.
Groups that perform second reviews have a measure of quality that may be of use within the group.
Groups that perform second reviews and fail to detect significant errors (<1 per 1000 cases) may have a problem with the sensitivity of their second reviews.
Finally, to measure or compare quality between groups, the following areas need further investigation:
Identify the optimal review methods that are applicable across many different groups.
Identify methods to measure and ensure the sensitivity of the second review.
Standardize criteria for review methods, definitions of errors, and optimal results.
Define expected ranges of performance.
Define methods to verify performance that falls outside that expected range.
We recommend that surgical pathology and cytology laboratories adopt a system of secondary timely case reviews that is suited to their practice and helps detect or prevent diagnostic interpretive errors.
For additional questions and comments, contact the Pathology and Laboratory Quality Center at firstname.lastname@example.org.
Supplemental digital content is available for this article at www.archivesofpathology.org in the January 2016 table of contents.
Authors' disclosures of potential conflicts of interest and author contributions are found in the appendix at the end of this article.