International Internet-Based Assessment of Observer Variability for Diagnostically Challenging Endometrial Biopsies

It is well recognized that endometrial carcinoma of endometrioid histology is often preceded by estrogen-driven glandular alterations harboring premalignant lesions. Originally recognized as adenomatous hyperplasia, these changes were codified by the 1994 World Health Organization (WHO) classification system in which 4 classes of hyperplasia were divided by architecture (complex and simple) and cytology (typical and atypical).1 Since that time, it has become clear that even expert pathologists are unable to reproducibly classify lesions based on cytologic atypia,2–4 and architectural features have been shown to have a cancer predictive value as important as cytology.5 

Previous studies have shown that the terminology of endometrial intraepithelial neoplasia (EIN) is a more accurate model of carcinogenesis and uses more precise diagnostic criteria for the identification of such precancers than the WHO classification system does.6,7 Subjective features that are diagnostic of EIN have been extrapolated from molecular and morphometric studies of lesions likely to progress to carcinoma.8 In contrast to the WHO classification, areas diagnosed as EIN are very precisely defined by 5 morphological criteria (Table 1). Furthermore, prior studies have shown that interobserver reproducibility of EIN versus non-EIN is good.8 However, despite the strict diagnostic criteria for EIN, at Beth Israel Deaconess Medical Center (Boston, Massachusetts), a number of benign mimics, such as endometrial polyps, secretory change, tubal metaplasia, and squamous morular metaplasia, continue to cause considerable diagnostic uncertainty.

We developed an online endometrial biopsy quiz to assess the causes of observer variability in the diagnosis of EIN in endometrial biopsies. Identification of these causes of variability is the first step toward a standardized and more unified criteria for diagnosis. Our goal was to share and evaluate an online EIN tutorial and to collect data on the use of EIN in the international pathology community. The quiz format we applied has been successfully used as an international platform for studies examining observer variability in such areas as urine and lung cytology, diagnosis of serrated polyps, and pleural invasion by lung cancer.9–12 

The cases were selected from a pool of intradepartmental, gynecologic consultations at Beth Israel Deaconess Medical Center. Cases were selected to illustrate common diagnostic dilemmas. The 18 quiz cases (Table 2) included 9 cases (50%) diagnosed as EIN, 4 (22%) as endometrial polyps, and 5 (28%) as benign changes (2 tubal metaplasia, 3 secretory changes). The EIN involved an endometrial polyp in 3 of the 9 EIN cases (33%), and extensive morular metaplasia was present in 1 endometrial polyp case and in 1 EIN case. One endometrial polyp contained complex mucinous metaplasia.

Each of the 18 quiz cases comprised 2 to 6 representative images, including at least one low-power and one high-power image, as well as relevant clinical information. Images are available at the quiz Web site (http://flexiform2.unibas.ch/formular.cfm?EID=1760; accessed May 6, 2010).

The registration form and the online quiz cases were constructed with the questionnaire tool FlexiForm created at the Computing Department of the University of Basel (Basel, Switzerland), which has been used previously.9–12 Pathologists worldwide were invited by e-mail and personal communication to register for the quiz. Many of those invited were members of the International Society of Gynecologic Pathology or registered users of the PATHO-L online listserv discussion group.

Participants had to choose 1 of the following 4 answers: EIN, polyp, benign endometrium (ie, proliferative, secretory, disordered, tubal metaplasia, or lower uterine segment), and adenocarcinoma. The authors' diagnoses and clinical follow-up were provided after answer submission. Answers of all participants were exported into an Excel file (Microsoft Office Excel 2003 software, Microsoft Corporation, Redmond, Washington) for evaluation. Free-marginal κ values were determined to assess levels of agreement.13 For the sake of analysis, the diagnosis of the author (J.D.M.) was considered correct. For analysis, the answers polyp and benign were combined into a single category. The EIN and adenocarcinoma answers were not combined.

On registration, participants were asked about such personal data as current position, number of years in practice, practice setting, country of practice, number of endometrial curettings signed-out per month, and current terminology used (EIN versus WHO). Before starting the quiz, all participants were provided the option of completing online tutorial materials that included an online education module (http://www.endometrium.org/EIN%20Central/EINcentral.htm, accessed May 6, 2010) and an unpublished review article written by one of the authors (J.L.H.) (http://eva.unibas.ch/download/44825-reviewEIN.pdf, accessed May 6, 2010).

There were 78 responses during a 4-month period. Participant characteristics are shown in Table 3. Agreement with the authors' diagnoses was largely unaffected by current position, time in practice, number of endometrial curettings signed-out per month, and current terminology used. Participants were from 13 different countries with most participants being from the United States (n  =  47; 60%) or Switzerland (n  =  10; 13%). The remaining participating countries included the following: Canada (n  =  3; 4%), China (n  =  3; 4%), Turkey (n  =  3; 4%), Italy (n  =  2; 3%), Japan (n  =  2; 3%), Spain (n  =  2; 3%), United Kingdom (n  =  2; 3%), Belgium (n  =  1; 1%), Paraguay (n  =  1; 1%), Puerto Rico (n  =  1; 1%), and Thailand (n  =  1; 1%). Of the 31 international participants, 5 (16%) reported using the EIN terminology (Turkey  =  1, Japan  =  2, Switzerland  =  2). Of the 47 United States participants, 15 (32%) reported using the EIN terminology.

An agreement of 100% was obtained by 2 surgical pathologists with an interest in gynecologic pathology. Both of these pathologists, 1 academic from the United States and 1 in private practice from Japan, have been practicing for more than 10 years, sign-out more than 20 endometrial curetting specimens per month, and use the EIN classification system. Participants who completed the online training module or read the review article before taking the quiz had a better mean agreement (60% agreement; n  =  52, 67% of all participants) compared with participants who did not (46% agreement; n  =  26; 33% of all participants), a difference that is not statistically significant. The significance of prequiz EIN training or of reading the review article was calculated by Fisher exact test for each question. Significance was only seen for quiz case 8 (P  =  .003) and quiz case 16 (P  =  .001), but we could not draw specific conclusions.

Overall, the percentage of agreement by each individual with the authors ranged from 17% to 100% (mean, 55%; κ  =  0.38). The mean percentage of agreement for all individual cases is listed in Table 2, and select images from the quiz are shown in Figures 1 through 4. The mean agreement for all 9 cases of EIN was 59% (κ  =  0.29); agreement for the 6 cases of EIN without metaplastic changes and not arising in a polyp was 63% (κ  =  0.36). After combining benign and polyp answers, participant diagnosing was best in cases of tubal metaplasia (87%; κ  =  0.69), secretory change (75%; κ  =  0.54), and endometrial polyps without metaplastic change (88%; κ  =  0.69).

The presence of metaplasia appeared to reduce consensus. Extensive morular metaplasia (28%; κ  =  −0.003) and mucinous metaplasia (38%; κ  =  0.34) were associated with the worst consensus rates. In quiz case 14 (Figure 1, A and B), only 33% diagnosed an endometrial polyp in the context of extensive squamous morular metaplasia. Most of the responses were EIN and carcinoma. In case 12 of the quiz (Figure 1, C and D), a case of EIN with extensive morular metaplasia, 42% (33 of 78) of participants diagnosed EIN, but 28% (22 of 78) overdiagnosed adenocarcinoma. Case 13 (Figure 2, A and B) was an endometrial polyp with mucinous metaplasia. This case was diagnosed as either benign or polyp by only 38% (30 of 78) of the participants. This case was also most commonly diagnosed as EIN or carcinoma. In cases without metaplastic change the consensus was excellent with 94% (73 of 78 responses) consensus for case 10 (secretory endometrium) and 96% (75 of 78 responses) consensus for quiz case 9 (a polyp without metaplastic change). In contrast, quiz case 4, showing only focal tubal metaplasia, was diagnosed as either benign or polyp by 82% of all participants (64 of 78).

The EIN arising in an endometrial polyp (Figure 3, A and B) also caused diagnostic confusion, and the 3 cases of EIN arising in a polyp had a mean agreement of 53%.

Secretory change caused significant confusion when confounded by tissue disruption and regional variation in gland architecture. Case 17, an EIN in a background of secretory endometrium (Figure 4, A and B), and case 11, a variable gland crowding in a secretory endometrium (Figure 4, C and D), showed agreement with the authors' diagnosis in only 23% and 42% of respondents (18 and 33 of 78 participants), respectively.

Although increasing in practice, the use of the EIN diagnostic schema is still not as common as that of the primary WHO classification system. Overall, 72% of the participants (56 of 78) use the WHO system. Interestingly, the mean agreement of the participants in the quiz was not influenced by the terminology they used in daily practice. Proponents of EIN emphasize the reproducibility of the diagnostic criteria.8 In this study, even though the quiz was composed entirely of cases that were considered diagnostically challenging, the reproducibility of EIN remained good and was perhaps even better than in studies evaluating reproducibility of the WHO classification system.3 The EIN schema is easy to learn, and prequiz training may have slightly improved performance.

This online quiz highlighted several features that complicate the diagnosis of EIN and reduce observer reproducibility. These were the presence of EIN without classic cytologic atypia, the presence of extensive metaplastic changes, EIN arising in a polyp, and EIN arising in a secretory background. In general, these are the same confounders found for hyperplasia, and special attention needs to be given to making the diagnosis of EIN in these scenarios.

The diagnostic difficulties caused by metaplastic lesions, when trying to distinguish them from EIN or hyperplasia, are well documented in the literature. One reason that metaplastic lesions are overdiagnosed as EIN has to do with the frequent coexistence of these lesions and with similar predisposing states (hyperestrinism). Carlson and Mutter14 described 83 sequential EIN lesions and found that 39 (47%) contained metaplastic changes (squamous morular, 18%; tubal secretory, 14%; and secretory, mucinous, or ciliated change, 5% each). However, these alterations can also be seen in benign endometrial biopsies and are occasionally extensive.15 The overlap in diagnostic criteria between the 2 lesions further confounds this distinction. Both EIN and metaplasia show cytologic alterations that are distinct from the uninvolved endometrium.

Morular metaplasia may also mimic the coalescence of glands seen in carcinoma.16 As an example, in quiz case 14 (Figure 1, A and B), the participants overdiagnosed adenocarcinoma (34%) or EIN (34%). This polyp shows extensive confluent squamous metaplasia and focal areas of gland crowding are difficult to measure. In areas of gland crowding, especially on high-power magnification, one can see that there is tubal metaplasia, suggesting a benign process to some of the participants. This case is similar to a case reported by Blaustein,16 in which a transvaginal ultrasound 4 years later was suggestive of endometrial polyps, but no additional tissue sampling was performed. Case 12 (Figure 1, C and D) shows morules in EIN mimicking the coalescence of glands seen in carcinoma; 28% of responders overdiagnosed carcinoma. The patient had recurrent bleeding, and an ultrasound showed changes suggestive of polyps 4 years later without tissue sampling. Another 2 years later, the patient was described as postmenopausal with no mention of hysterectomy.

Mucinous metaplasia is associated with concurrent neoplasia only when it is present in a complex-rigid glandular architecture, in a filiform growth pattern, or with cytologic atypia.17 The significance of these findings in a polyp, as illustrated in case 13 (Figure 2, A and B), is controversial.18 Case 13, diagnosed as EIN by 39% of participants (30 of 78), and as adenocarcinoma by 23% (18 of 78), is a glandular proliferation in a polyp that does not form a discrete focus with sufficient crowding for EIN but has intraglandular papillary projections. Given a lack of clinical follow-up and molecular data in the literature, such a case, in routine practice, would likely be diagnosed descriptively with a suggestion for follow-up sampling. That was not a choice on the quiz, and participants were unwilling to let the case pass as benign. Subsequent curetting on this 61-year-old woman showed similar mucinous metaplasia, and the patient stopped bleeding. She was treated for stage 1 breast cancer at 1 year out and is alive and well with many orthopedic complaints 9 years later.

Cases with EIN confined to a polyp (Figure 3, A and B) were recognized by most of the participants. Curetted endometrial polyps are characterized by large, tissue fragments containing endometrial glands of variable size, altered stroma, and thick-walled vessels. The diagnosis of EIN within a polyp is difficult because of the inherent background variation in architecture and cytology. An EIN can only be diagnosed in cases that have clearly defined, geographic clustering with cytologic distinction from the background. However, some cases will not be apparent on routine stain.19 The distinction of an endometrial polyp from an anovulatory or another benign endometrium is sometimes obscured by fragmentation. This is apparent in case 18, where agreement was improved by combining the polyp and benign categories.

Participants generally performed well on cases of tubal metaplasia and secretory endometrium, with some overdiagnosis of EIN. Although tissue disruption and regional variation in secretory change can be confusing, EIN in a secretory background typically stands out as a discrete focus of noncycling (inactive or proliferative type) glands. However, in occasional cases, there was disagreement even in this setting (case 17; Figure 4, A and B). It is not entirely clear why agreement was so poor in this example, but it is likely due to the limited image format of presentation. The lesion is present in a single separate tissue fragment. The other fragments show secretory endometrium, but no single image shows that contrast, and only the EIN is shown at high-power magnification. Conversely, in case 11, secretory change in a fragmented, architecturally disordered endometrium was overdiagnosed as EIN. A single, intact fragment shows randomly arranged, irregularly shaped, crowded glands (Figure 4, C), but the cytology is not distinct from the secretory background. The infoldings and secretions within the glands are a clue to the diagnosis at low-power magnification, but a high-power image (Figure 4, D) was not provided in the quiz. Again, the poor results may be due to the presentation. A subsequent hysterectomy was diagnosed with focally complex, endometrial hyperplasia with mild cytologic atypia.

Although the quiz was constructed using a collection of cases that were considered difficult by the authors, the diagnostic choices were limited. We did not include a category for indeterminate for EIN among the quiz answers to simplify the consensus analysis, which may have resulted in a diagnostic bias. We forced participants to make a decision because we wanted to see how specific, metaplastic changes were interpreted in a situation where EIN terminology had be to used. No attempt was made to try to estimate the degree to which diagnostic bias was created by the limited categories.

In clinical practice, some of these cases would likely have been handled with a descriptive note or designation as indeterminate for EIN, with suggestions for follow-up, including resampling. In fact, Huang et al20 have recently shown that 23% of such cases will be followed by a histologic diagnosis of neoplasia in a subsequent sampling. However, they describe these cases as representing interpretively difficult samples rather than as a new diagnostic category within the EIN terminology. The indeterminate category cannot be used to establish a specific risk of cancer, but additional sampling surely increases the sensitivity for cancer detection. The EIN histologic diagnostic criteria are based on extrapolation from computerized morphometry, and all features (Table 1) must be present for a diagnosis.7 Diagnosis of EIN is not made on a point-based scoring index, does not have subcategories of cancer risk, and is not graded.

The general lack of agreement for some cases does not necessarily reflect the actual, daily performance of practicing pathologists where noncategoric diagnoses and descriptive notes are common. The authors' diagnosis was used as a basis of comparison and a focal point of discussion, not as a gold standard. Our intention was to illustrate areas of consensus and variation rather than to dictate a correct answer. Clinical follow-up is not a good gold standard for neoplasia in these cases because low-grade endometrioid neoplasms can be indolent, especially when confined to a polyp, and may even regress or be cured by curettage.21–23 For example, this may have occurred for case 17, in which EIN was identified in a background of secretory endometrium, but subsequent follow-up showed only secretory endometrium.

Perhaps the major limitation of this study was the limited format and variable number of selected case images. The number of images and magnifications available to participants varied for each case. Images were selected to contain all relevant diagnostic areas. Some cases required up to 6 images, whereas other cases required only 2. This potentially could have introduced bias, with some participants interpreting fewer images as being more straightforward and more images as being more difficult. As a result, participants could have upgraded or downgraded their responses based on the number of images available for each case. To facilitate participation and to prevent attrition, only the minimal number of images was included for each case, a method that has been used in prior online quizzes.11 As virtual microscopy becomes a more widely available technology, the use of whole slide imaging in the future could remove this source of bias and potentially improve agreement.

In conclusion, this international, online quiz illustrated common problems in diagnosing EIN in endometrial biopsies. In particular, those cases with squamous morular metaplasia, those with mucinous metaplasia, and those arising within endometrial polyps likely require consensus review.