Validation of a Cytotechnologist Manual Counting Service for the Ki67 Index in Neuroendocrine Tumors of the Pancreas and Gastrointestinal Tract

Slides from 18 archived cases of gastrointestinal or pancreatic NET from a single institution were selected to represent a wide spectrum of Ki67 indexes. Immunohistochemistry for Ki67 was performed using the Dako (Agilent Technologies, Santa Clara, California) MIB1 clone (1:400 dilution) with a Ventana Benchmark automated system (Ventana Medical System Inc, Tucson, Arizona). The Ultra Cell Conditioner 1 solution (Ventana) was applied for 32 minutes for antigen retrieval and the iView DAB detection kit (Ventana) was used. Human tonsil tissue, expected to have Ki67 staining in 80% to 90% of germinal center cells, served as a same-slide positive control.

Digital images of the Ki67 immunostains were obtained using the Aperio Scanscope XT, scanned at ×200 magnification (Leica Biosystems, Wetzlar, Germany), and captured by a Leica Imagescope version 12.2 (Leica Biosystems). Images at ×200 magnification were printed in color at a size of 8 inches in height by 10 inches in width, at 96 dots per inch resolution. Each digital image was estimated to include an area of at least 1000 cells for assessment, corresponding to approximately a 0.5 mm² area of tissue on the slide. Images from 18 cases were selected and 2 were printed in duplicate and included in a package of 20 photos to assess intrarater variability. All study participants were blinded to the patient identifiers, specimen site, prior pathology reports, and Ki67 indexes reported by other raters.

Four pathologists performed eyeball estimates of the Ki67 index to the nearest 1% for glass slides on the area they identified as a hot spot. They were instructed not to perform any manual counting during the eyeball estimate. The time taken for each eyeball estimate was recorded. Following a break of at least 1 week, the same 4 pathologists provided eyeball estimates of the Ki67 index on 20 printed digital images, with the time taken recorded. Following another break of at least 2 weeks, 3 of the 4 pathologists performed a manual count on the same package of 20 digital image prints, received in a different random order (1 of the original 4 pathologists did not complete the manual count). The mean of the 3 pathologist manual counts from the digital images was considered the gold standard Ki67 result for the study.

Three cytotechnologists, each with more than 10 years of experience, were recruited to provide manual counts for the same set of 20 digital image prints. Cytotechnologists were given a 45-minute tutorial by one of the participating pathologists prior to receiving the image packages. The tutorial included examples of Ki67-stained glass slides reviewed at a multiheaded microscope and a demonstration of manual counting on a printed image with the method of striking out negative cells and circling positive cells. A written instruction sheet (summarized in the Table) outlining the counting strategies was included with the package of digital images. Cytotechnologists were instructed to count tumor cells and to exclude stromal cells and lymphocytes.

The WHO 2010 grade was determined based on the Ki67 index alone. The mitotic index was not included. The tumor was labeled as grade 1 if the Ki67 index was 0% to 2.99%, grade 2 for 3% to 20%, and grade 3 for greater than 20%.⁴ 

Intraclass correlation coefficient (ICC) was used to assess the interrater reliability of the Ki67 index. The ICC was calculated using a single-measure 2-way mixed model with absolute agreement. The ICC reflects a scale of reliability measurement of a group of raters, where 1.0 corresponds with perfect reliability and 0.0 represents no reliability. The Cohen κ was used to evaluate WHO tumor grade agreement between and within groups. Based on the Landis and Koch¹²  guidelines, κ values may be interpreted as 0 to 0.2 representing slight agreement, 0.2 to 0.4 fair, 0.4 to 0.6 moderate, 0.6 to 0.8 substantial, and more than 0.8 almost perfect agreement. All analyses were carried out using SPSS 22.0 (IBM, Armonk, New York) software.

All 4 Ki67 assessment methods demonstrated excellent interrater reliability, as judged using the intraclass correlation coefficient statistic. The ICCs recorded for each group were as follows: pathologist manual count ICC = 0.99 (95% CI, 0.98–0.99), cytotechnologist manual count ICC = 0.97 (95% CI, 0.93–0.99), pathologist glass slide eyeball estimate ICC = 0.97 (95% CI, 0.94–0.99), and pathologist photomicrograph eyeball estimate ICC = 0.97 (95% CI, 0.93–0.98). There were high levels of reliability between the cytotechnologist manual count and the pathologist manual count (ICC = 0.99; 95% CI, 0.99–0.99) and between the pathologist manual count and the pathologist image eyeball estimate (ICC = 0.99; 95% CI, 0.98–0.99) using this statistic.

The Ki67 proliferation index for the 18 cases with digital photomicrographs ranged from 0.88% to 92.95% (mean of 12.98%) according to the mean count of 3 pathologist reviewers. The mean Ki67 index from pathologist manual counts was used as the gold standard to assign each case a corresponding WHO tumor grade. Eleven of 18 cases (61%) were designated WHO grade 1, 5 cases (28%) were WHO grade 2, and 2 (11%) were WHO grade 3. The photomicrographs included a mean of 1765 cells per image (range, 99–2550 cells). The interrater reliability of the pathologists' WHO grade assignments by manual count was substantial, as demonstrated by the Cohen κ for each pair of pathologists (κ values 0.76, 0.73, and 0.66).

The level of agreement for tumor grade between pairs of pathologist glass slide eyeball estimates was moderate to substantial (κ values ranged from 0.42 to 0.63). The agreement was fair to moderate for pairs of pathologist photomicrograph eyeball estimates (κ values ranged from 0.39 to 0.60). Tumor grade agreement between the gold standard mean pathologist manual count and the mean pathologist image eyeball estimate was fair (κ value of 0.31; 95% CI, 0.030–0.60). In 9 of 20 cases (45%), there was a lack of agreement in WHO tumor grade between the eyeball estimate and the manual count. In all discordant cases, the pathologist eyeball estimate overcalled (upgraded) the tumor grade. Five cases classified as grade 1 by the gold standard manual count method were called grade 2 based on image eyeball estimation. Similarly, 4 cases that were WHO grade 2 by manual counts were called grade 3 by image eyeball estimation. Two examples are illustrated in Figure 1, a and b.

Cohen κ values calculated for mean cytotechnologist manual count versus mean pathologist manual count demonstrated almost perfect agreement in classifying tumor grade (κ value of 0.91; 95% CI, 0.70–1.0). Of the 20 images assessed, there was only 1 grade disagreement between the 2 groups (illustrated in Figure 2, a). In this case, the average Ki67 index by pathologist manual count was 2.94%, whereas the average Ki67 index by cytotechnologist manual count was 3.14%.

In all 20 images, 2 of the 3 cytotechnologist reviewers achieved perfect interrater reliability (κ value 1.0) in tumor grading. The third cytotechnologist assigned discordant tumor grades in 9 of 20 cases (45%), corresponding to a κ value of 0.32 when compared with either of the other 2 cytotechnologists. Each of these discrepancies resulted in an overcall of the WHO grade. We reviewed all paper photomicrographs counted by the cytotechnologists and compared them with those of the pathologists. Review of the manually counted photomicrographs revealed an overestimation of the Ki67 index by a single cytotechnologist due to erroneous counting of cells with nonspecific cytoplasmic staining and counting fewer negative tumor cells. These findings are illustrated in Figure 3, a and b. We noted that all 3 cytotechnologists displayed an excellent ability to distinguish tumor cells from stromal cells and lymphocytes.

Each package of photomicrographs provided to study raters included duplicate images from 2 cases to assess intraobserver variability among the 3 paper-based methods (pathologist eyeball estimate, pathologist manual count, and cytotechnologist manual count). By the gold standard pathologist manual count, the first case was evaluated to have a mean Ki67 index of 1.5% (grade 1) and the other case had an index of 19% (grade 2). There was 1 grade discrepancy in a pathologist eyeball estimate in 1 of these 2 cases, but no grade discrepancies in cytotechnologist or pathologist manual counts.

The mean time cytotechnologists spent on performing manual counts for each case was 18.8 minutes. Overall, pathologists required slightly less time for their manual counts, with a mean of 17.0 minutes per case. Pathologist eyeball estimates using glass slides took an average of 57.0 seconds per case, whereas the digital image eyeball estimate required the least time, with an average of 42.0 seconds per case.

Our results support the opinion that grading NETs using an eyeball estimate of the Ki67 index by pathologists, although convenient, is error prone. Eyeballing tends to upgrade tumors, which occurred in 45% of the eyeball estimates in our study. Accurate counts in Ki67 are important: a very small difference in the Ki67 index can translate into inconsistencies when assigning WHO tumor grade, as shown in Figure 2. Prior studies have also shown poor concordance between pathologist eyeball estimates and manual counts.^8,9  Manual counting of printed photographs is increasingly considered to be the most accurate approach to Ki67 index determination and NET grading.⁶  However, this practice requires a major investment of pathologist time for a small part of a single case, a mean of 17 minutes per case in our study for the count alone, not including the time needed to take and print a photograph. The time required to provide a manual count for the Ki67 index makes the method unpopular despite the improved accuracy compared with eyeball estimation.

Accurate, time-saving alternatives to pathologist manual counting are of interest. The main alternative approach that has been investigated in the literature is digital image analysis.^6,8,10,11  Although cost was once an issue, free Internet-based image analyzers such as ImmunoRatio¹⁰  are currently available online. Digital image analysis certainly shows some promise in terms of accuracy, but the promised savings in pathologist time are less clear. In addition to training to use the image analyzer program, for each case, the digital image analyzers require pathologist time to photograph slides and load the image into the analysis software. Personal experience suggests that these programs often work best if an additional step of photo editing is taken to improve color and contrast. Image analysis software continues to have pitfalls such as discriminating tumor cells from stromal cells and lymphocytes, differentiating hemosiderin and nonspecific cytoplasmic staining from positive nuclei, and interpreting different staining intensities with consistent results.⁶  At least 1 of the digital image analysis studies notes that a step of manually excluding lymphocytes and stromal cells from the computer calculation is required, which adds another time-consuming step.⁸ 

We undertook this study because we anticipated that manual Ki67 counts by cytotechnologists would probably have similar accuracy to pathologist counts, and importantly, this would obviate the need for pathologists to work through all the steps involved in a digital image analysis or perform a time-consuming manual count. Having cytotechnologists perform manual Ki67 index counts for NET grading is not an approach that has been previously validated in the literature by direct comparison with pathologist reviewers. However, there are studies reporting participation of cytotechnologists in the interpretation of quantitative immunohistochemistry.^5,13  The appeal of recruiting cytotechnologists for this role is their high level of skill in cytomorphology, such that minimal training is required for this group to recognize nuclei and distinguish neoplastic and nonneoplastic cells. The participating cytotechnologists in our study provided highly accurate Ki67 manual counts. There was only a single grade discordance between the mean pathologist and mean cytotechnologist manual count in 20 cases (illustrated in Figure 2, a). The pathologist manual count in this case was 2.94%, compared with the cytotechnologist manual count of 3.14%, illustrating how a very small difference in manual count calculations may translate to inconsistencies when assigning WHO tumor grade. Two of the 3 cytotechnologists achieved perfect WHO tumor grade concordance for all 20 cases when compared with the gold standard pathologist manual count. One of the 3 cytotechnologists did tend to overcall the Ki67 index and thus upgrade the tumor (in 9 of 20 cases). The main issues for this cytotechnologist were counting nonspecific cytoplasmic staining as positive and missing some negative tumor nuclei. The quality of the immunohistochemical stain and the quality of the image were probably factors, although these did not prove to be a major factor for the other cytotechnologists or the pathologists. This could be addressed through further emphasis on these issues in the initial training session and through pathologist review of the manually counted images (where the cytotechnologist has circled positive cells and struck out negative cells) for feedback on an ongoing basis.

Applying an overlying grid to the printed images has been shown to improve both accuracy of counts and ease of counting.¹⁴  Reducing the number of cells per image has also been shown to improve accuracy while decreasing time spent counting.¹⁵  Our study photomicrographs included an average of 1765 cells on a single sheet of paper per case. Both the pathologist and cytotechnologist reviewers in our study suggested that this large number of cells per page contributed to the challenge of accurate manual counting in the study materials. This approach was largely done to reduce color printing costs in a study that had multiple reviewers. When implementing the cytotechnologist counting service, we suggest printing and counting multiple images with 500 cells or less per image for each case. We also suggest having the pathologist mark the hot spot on the slide. The cytotechnologist assigned to the quantitative immunohistochemistry service will photograph and print the image for manual counting. The pathologist will have the opportunity to review the manually counted picture prior to sign-out.

In our opinion, applying cytotechnologist skills to this expanded role within surgical pathology would be of great benefit for pathologists and cytotechnologists alike. As cervical screening guidelines change with less frequent screening intervals and more widespread adoption of human papillomavirus testing algorithms, the cervical cytology workload is decreasing. Quantitative immunohistochemistry analysis is a role that can potentially increase the cytotechnologist scope of practice. During a poststudy debriefing session with our 3 cytotechnologists, all expressed enthusiasm for incorporating manual Ki67 counts in their professional practice.

Our study demonstrates that cytotechnologists can accurately perform manual counts of Ki67 for NET grading, and in turn reduce pathologist workload. This approach has the potential to expand the cytotechnologist's role in the laboratory and improve the accuracy of results used in clinical decision making.

We are very grateful to Carolyn Lohnes, MLT, and Heather MacIntyre, MLT, cytotechnologists at the Queen Elizabeth II Health Sciences Centre, who performed manual Ki67 counts on digital image prints for this study.