Context.—

Pathologists routinely assess Ki67 immunohistochemistry to grade gastrointestinal and pancreatic neuroendocrine tumors. Unfortunately, manual counts of the Ki67 index are very time consuming and eyeball estimation has been criticized as unreliable. Manual Ki67 counts performed by cytotechnologists could potentially save pathologist time and improve accuracy.

Objective.—

To assess the concordance between manual Ki67 index counts performed by cytotechnologists versus eyeball estimates and manual Ki67 counts by pathologists.

Design.—

One Ki67 immunohistochemical stain was retrieved from each of 18 archived gastrointestinal or pancreatic neuroendocrine tumor resections. We compared pathologists' Ki67 eyeball estimates on glass slides and printed color images with manual counts performed by 3 cytotechnologists and gold standard manual Ki67 index counts by 3 pathologists.

Results.—

Tumor grade agreement between pathologist image eyeball estimate and gold standard pathologist manual count was fair (κ = 0.31; 95% CI, 0.030–0.60). In 9 of 20 cases (45%), the mean pathologist eyeball estimate was 1 grade higher than the mean pathologist manual count. There was almost perfect agreement in classifying tumor grade between the mean cytotechnologist manual count and the mean pathologist manual count (κ = 0.910; 95% CI, 0.697–1.00). In 20 cases, there was only 1 grade disagreement between the 2 methods. Eyeball estimation by pathologists required less than 1 minute, whereas manual counts by pathologists required a mean of 17 minutes per case.

Conclusions.—

Eyeball estimation of the Ki67 index has a high rate of tumor grade misclassification compared with manual counting. Cytotechnologist manual counts are accurate and save pathologist time.

Neuroendocrine tumors (NETs) of the gastrointestinal tract and pancreas are uncommon neoplasms with marked heterogeneity in clinical behavior.1,2  Ki67 immunohistochemistry has emerged as an important prognostic indicator and is a key determinant of NET grade based on the 2010 World Health Organization (WHO) classification of tumors of the gastrointestinal tract and pancreas.3,4 

Pathologists report the Ki67 index by evaluating the proportion of tumor cells with greatest nuclear labeling by the Ki67 immunostain in a hot-spot microscopic field containing at least 500 to 2000 tumor cells.4  Although time consuming, a pathologist manual count of the Ki67 index using a printed photomicrograph is increasingly considered the gold standard approach.5,6  However, in routine practice pathologists more typically report an eyeball estimate of the Ki67 index from the glass slide as a timesaving measure.7  Unfortunately, studies have shown that the eyeball estimate method does not accurately reflect NET grade when compared with manual counting.8,9  Several authors have proposed digital image analysis as an alternative approach to calculating the Ki67 index.6,8,10,11  However, this technology has limitations, including time required to acquire digital images and use the software.

Cytotechnologists are highly skilled cytomorphologists trained to distinguish neoplastic versus nonneoplastic cell populations. Given their expertise, we suspected that cytotechnologists could accurately do a manual count of the Ki67 index in surgical pathology cases with minimal training, saving pathologist time. This study formally tests that hypothesis.

MATERIALS AND METHODS

Case Selection and Ki67 Immunohistochemistry

Slides from 18 archived cases of gastrointestinal or pancreatic NET from a single institution were selected to represent a wide spectrum of Ki67 indexes. Immunohistochemistry for Ki67 was performed using the Dako (Agilent Technologies, Santa Clara, California) MIB1 clone (1:400 dilution) with a Ventana Benchmark automated system (Ventana Medical System Inc, Tucson, Arizona). The Ultra Cell Conditioner 1 solution (Ventana) was applied for 32 minutes for antigen retrieval and the iView DAB detection kit (Ventana) was used. Human tonsil tissue, expected to have Ki67 staining in 80% to 90% of germinal center cells, served as a same-slide positive control.

Digital Images

Digital images of the Ki67 immunostains were obtained using the Aperio Scanscope XT, scanned at ×200 magnification (Leica Biosystems, Wetzlar, Germany), and captured by a Leica Imagescope version 12.2 (Leica Biosystems). Images at ×200 magnification were printed in color at a size of 8 inches in height by 10 inches in width, at 96 dots per inch resolution. Each digital image was estimated to include an area of at least 1000 cells for assessment, corresponding to approximately a 0.5 mm2 area of tissue on the slide. Images from 18 cases were selected and 2 were printed in duplicate and included in a package of 20 photos to assess intrarater variability. All study participants were blinded to the patient identifiers, specimen site, prior pathology reports, and Ki67 indexes reported by other raters.

Pathologist Counts

Four pathologists performed eyeball estimates of the Ki67 index to the nearest 1% for glass slides on the area they identified as a hot spot. They were instructed not to perform any manual counting during the eyeball estimate. The time taken for each eyeball estimate was recorded. Following a break of at least 1 week, the same 4 pathologists provided eyeball estimates of the Ki67 index on 20 printed digital images, with the time taken recorded. Following another break of at least 2 weeks, 3 of the 4 pathologists performed a manual count on the same package of 20 digital image prints, received in a different random order (1 of the original 4 pathologists did not complete the manual count). The mean of the 3 pathologist manual counts from the digital images was considered the gold standard Ki67 result for the study.

Cytotechnologist Manual Counts

Three cytotechnologists, each with more than 10 years of experience, were recruited to provide manual counts for the same set of 20 digital image prints. Cytotechnologists were given a 45-minute tutorial by one of the participating pathologists prior to receiving the image packages. The tutorial included examples of Ki67-stained glass slides reviewed at a multiheaded microscope and a demonstration of manual counting on a printed image with the method of striking out negative cells and circling positive cells. A written instruction sheet (summarized in the Table) outlining the counting strategies was included with the package of digital images. Cytotechnologists were instructed to count tumor cells and to exclude stromal cells and lymphocytes.

Instructions to Cytotechnologists for Counting the Ki67 Index

Instructions to Cytotechnologists for Counting the Ki67 Index
Instructions to Cytotechnologists for Counting the Ki67 Index

WHO Grading

The WHO 2010 grade was determined based on the Ki67 index alone. The mitotic index was not included. The tumor was labeled as grade 1 if the Ki67 index was 0% to 2.99%, grade 2 for 3% to 20%, and grade 3 for greater than 20%.4 

Statistics

Intraclass correlation coefficient (ICC) was used to assess the interrater reliability of the Ki67 index. The ICC was calculated using a single-measure 2-way mixed model with absolute agreement. The ICC reflects a scale of reliability measurement of a group of raters, where 1.0 corresponds with perfect reliability and 0.0 represents no reliability. The Cohen κ was used to evaluate WHO tumor grade agreement between and within groups. Based on the Landis and Koch12  guidelines, κ values may be interpreted as 0 to 0.2 representing slight agreement, 0.2 to 0.4 fair, 0.4 to 0.6 moderate, 0.6 to 0.8 substantial, and more than 0.8 almost perfect agreement. All analyses were carried out using SPSS 22.0 (IBM, Armonk, New York) software.

RESULTS

Ki67 Index Interrater Reliability by Intraclass Correlation Coefficient

All 4 Ki67 assessment methods demonstrated excellent interrater reliability, as judged using the intraclass correlation coefficient statistic. The ICCs recorded for each group were as follows: pathologist manual count ICC = 0.99 (95% CI, 0.98–0.99), cytotechnologist manual count ICC = 0.97 (95% CI, 0.93–0.99), pathologist glass slide eyeball estimate ICC = 0.97 (95% CI, 0.94–0.99), and pathologist photomicrograph eyeball estimate ICC = 0.97 (95% CI, 0.93–0.98). There were high levels of reliability between the cytotechnologist manual count and the pathologist manual count (ICC = 0.99; 95% CI, 0.99–0.99) and between the pathologist manual count and the pathologist image eyeball estimate (ICC = 0.99; 95% CI, 0.98–0.99) using this statistic.

Gold Standard Pathologist Manual Counts for the Study Population

The Ki67 proliferation index for the 18 cases with digital photomicrographs ranged from 0.88% to 92.95% (mean of 12.98%) according to the mean count of 3 pathologist reviewers. The mean Ki67 index from pathologist manual counts was used as the gold standard to assign each case a corresponding WHO tumor grade. Eleven of 18 cases (61%) were designated WHO grade 1, 5 cases (28%) were WHO grade 2, and 2 (11%) were WHO grade 3. The photomicrographs included a mean of 1765 cells per image (range, 99–2550 cells). The interrater reliability of the pathologists' WHO grade assignments by manual count was substantial, as demonstrated by the Cohen κ for each pair of pathologists (κ values 0.76, 0.73, and 0.66).

Concordance in WHO Grades Between Different Observers

The level of agreement for tumor grade between pairs of pathologist glass slide eyeball estimates was moderate to substantial (κ values ranged from 0.42 to 0.63). The agreement was fair to moderate for pairs of pathologist photomicrograph eyeball estimates (κ values ranged from 0.39 to 0.60). Tumor grade agreement between the gold standard mean pathologist manual count and the mean pathologist image eyeball estimate was fair (κ value of 0.31; 95% CI, 0.030–0.60). In 9 of 20 cases (45%), there was a lack of agreement in WHO tumor grade between the eyeball estimate and the manual count. In all discordant cases, the pathologist eyeball estimate overcalled (upgraded) the tumor grade. Five cases classified as grade 1 by the gold standard manual count method were called grade 2 based on image eyeball estimation. Similarly, 4 cases that were WHO grade 2 by manual counts were called grade 3 by image eyeball estimation. Two examples are illustrated in Figure 1, a and b.

Figure 1

This figure illustrates Ki67 immunohistochemistry from 2 cases in which there was discordance in the tumor grade between the pathologist eyeball estimate and the manual counts. a, The tumor was labeled as World Health Organization (WHO) grade 2 based on eyeball estimates, but proved to be grade 1 on manual counts by both pathologists and cytotechnologists. b, The case was labeled as WHO grade 3 based on eyeball estimates by pathologists, but was determined to be grade 2 on cytologist and pathologist manual counts (Ki67 immunohistochemistry, original magnification ×200).

Figure 1

This figure illustrates Ki67 immunohistochemistry from 2 cases in which there was discordance in the tumor grade between the pathologist eyeball estimate and the manual counts. a, The tumor was labeled as World Health Organization (WHO) grade 2 based on eyeball estimates, but proved to be grade 1 on manual counts by both pathologists and cytotechnologists. b, The case was labeled as WHO grade 3 based on eyeball estimates by pathologists, but was determined to be grade 2 on cytologist and pathologist manual counts (Ki67 immunohistochemistry, original magnification ×200).

Cohen κ values calculated for mean cytotechnologist manual count versus mean pathologist manual count demonstrated almost perfect agreement in classifying tumor grade (κ value of 0.91; 95% CI, 0.70–1.0). Of the 20 images assessed, there was only 1 grade disagreement between the 2 groups (illustrated in Figure 2, a). In this case, the average Ki67 index by pathologist manual count was 2.94%, whereas the average Ki67 index by cytotechnologist manual count was 3.14%.

Figure 2

a, World Health Organization (WHO) tumor grade according to the mean cytotechnologist manual count (blue bars) and the gold standard pathologist manual count (red bars) for each of 20 cases. There is a single discordance (case 3). b, Tumor grade based on mean pathologist eyeball estimate for the digital image from each case (blue bars) and the gold standard pathologist manual count (red bars). There are 9 discordant cases by eyeball estimate, all of which upgrade the tumor.

Figure 2

a, World Health Organization (WHO) tumor grade according to the mean cytotechnologist manual count (blue bars) and the gold standard pathologist manual count (red bars) for each of 20 cases. There is a single discordance (case 3). b, Tumor grade based on mean pathologist eyeball estimate for the digital image from each case (blue bars) and the gold standard pathologist manual count (red bars). There are 9 discordant cases by eyeball estimate, all of which upgrade the tumor.

In all 20 images, 2 of the 3 cytotechnologist reviewers achieved perfect interrater reliability (κ value 1.0) in tumor grading. The third cytotechnologist assigned discordant tumor grades in 9 of 20 cases (45%), corresponding to a κ value of 0.32 when compared with either of the other 2 cytotechnologists. Each of these discrepancies resulted in an overcall of the WHO grade. We reviewed all paper photomicrographs counted by the cytotechnologists and compared them with those of the pathologists. Review of the manually counted photomicrographs revealed an overestimation of the Ki67 index by a single cytotechnologist due to erroneous counting of cells with nonspecific cytoplasmic staining and counting fewer negative tumor cells. These findings are illustrated in Figure 3, a and b. We noted that all 3 cytotechnologists displayed an excellent ability to distinguish tumor cells from stromal cells and lymphocytes.

Figure 3

This figure illustrates the 2 pitfalls in manual counting that we identified with one cytotechnologist. a, The image counted by the cytotechnologist. b, The same image counted by a pathologist. Both use the same method of circling positive cells and striking out negative tumor cells. The red arrows show 2 areas where the cytotechnologist labeled nonspecific cytoplasmic staining as positive. The green ellipses surround 2 spots where the cytotechnologist did not see and strike out negative cells, which have weak counterstaining. This resulted in the cytotechnologist upgrading this case compared with the gold standard pathologist manual count. Subjectively, this was an unusually challenging case because of a high amount of nonspecific cytoplasmic staining (Ki67 immunohistochemistry, original magnification ×200).

Figure 3

This figure illustrates the 2 pitfalls in manual counting that we identified with one cytotechnologist. a, The image counted by the cytotechnologist. b, The same image counted by a pathologist. Both use the same method of circling positive cells and striking out negative tumor cells. The red arrows show 2 areas where the cytotechnologist labeled nonspecific cytoplasmic staining as positive. The green ellipses surround 2 spots where the cytotechnologist did not see and strike out negative cells, which have weak counterstaining. This resulted in the cytotechnologist upgrading this case compared with the gold standard pathologist manual count. Subjectively, this was an unusually challenging case because of a high amount of nonspecific cytoplasmic staining (Ki67 immunohistochemistry, original magnification ×200).

Intraobserver Variation

Each package of photomicrographs provided to study raters included duplicate images from 2 cases to assess intraobserver variability among the 3 paper-based methods (pathologist eyeball estimate, pathologist manual count, and cytotechnologist manual count). By the gold standard pathologist manual count, the first case was evaluated to have a mean Ki67 index of 1.5% (grade 1) and the other case had an index of 19% (grade 2). There was 1 grade discrepancy in a pathologist eyeball estimate in 1 of these 2 cases, but no grade discrepancies in cytotechnologist or pathologist manual counts.

Time Spent With Each Ki67 Assessment Method

The mean time cytotechnologists spent on performing manual counts for each case was 18.8 minutes. Overall, pathologists required slightly less time for their manual counts, with a mean of 17.0 minutes per case. Pathologist eyeball estimates using glass slides took an average of 57.0 seconds per case, whereas the digital image eyeball estimate required the least time, with an average of 42.0 seconds per case.

DISCUSSION

Our results support the opinion that grading NETs using an eyeball estimate of the Ki67 index by pathologists, although convenient, is error prone. Eyeballing tends to upgrade tumors, which occurred in 45% of the eyeball estimates in our study. Accurate counts in Ki67 are important: a very small difference in the Ki67 index can translate into inconsistencies when assigning WHO tumor grade, as shown in Figure 2. Prior studies have also shown poor concordance between pathologist eyeball estimates and manual counts.8,9  Manual counting of printed photographs is increasingly considered to be the most accurate approach to Ki67 index determination and NET grading.6  However, this practice requires a major investment of pathologist time for a small part of a single case, a mean of 17 minutes per case in our study for the count alone, not including the time needed to take and print a photograph. The time required to provide a manual count for the Ki67 index makes the method unpopular despite the improved accuracy compared with eyeball estimation.

Accurate, time-saving alternatives to pathologist manual counting are of interest. The main alternative approach that has been investigated in the literature is digital image analysis.6,8,10,11  Although cost was once an issue, free Internet-based image analyzers such as ImmunoRatio10  are currently available online. Digital image analysis certainly shows some promise in terms of accuracy, but the promised savings in pathologist time are less clear. In addition to training to use the image analyzer program, for each case, the digital image analyzers require pathologist time to photograph slides and load the image into the analysis software. Personal experience suggests that these programs often work best if an additional step of photo editing is taken to improve color and contrast. Image analysis software continues to have pitfalls such as discriminating tumor cells from stromal cells and lymphocytes, differentiating hemosiderin and nonspecific cytoplasmic staining from positive nuclei, and interpreting different staining intensities with consistent results.6  At least 1 of the digital image analysis studies notes that a step of manually excluding lymphocytes and stromal cells from the computer calculation is required, which adds another time-consuming step.8 

We undertook this study because we anticipated that manual Ki67 counts by cytotechnologists would probably have similar accuracy to pathologist counts, and importantly, this would obviate the need for pathologists to work through all the steps involved in a digital image analysis or perform a time-consuming manual count. Having cytotechnologists perform manual Ki67 index counts for NET grading is not an approach that has been previously validated in the literature by direct comparison with pathologist reviewers. However, there are studies reporting participation of cytotechnologists in the interpretation of quantitative immunohistochemistry.5,13  The appeal of recruiting cytotechnologists for this role is their high level of skill in cytomorphology, such that minimal training is required for this group to recognize nuclei and distinguish neoplastic and nonneoplastic cells. The participating cytotechnologists in our study provided highly accurate Ki67 manual counts. There was only a single grade discordance between the mean pathologist and mean cytotechnologist manual count in 20 cases (illustrated in Figure 2, a). The pathologist manual count in this case was 2.94%, compared with the cytotechnologist manual count of 3.14%, illustrating how a very small difference in manual count calculations may translate to inconsistencies when assigning WHO tumor grade. Two of the 3 cytotechnologists achieved perfect WHO tumor grade concordance for all 20 cases when compared with the gold standard pathologist manual count. One of the 3 cytotechnologists did tend to overcall the Ki67 index and thus upgrade the tumor (in 9 of 20 cases). The main issues for this cytotechnologist were counting nonspecific cytoplasmic staining as positive and missing some negative tumor nuclei. The quality of the immunohistochemical stain and the quality of the image were probably factors, although these did not prove to be a major factor for the other cytotechnologists or the pathologists. This could be addressed through further emphasis on these issues in the initial training session and through pathologist review of the manually counted images (where the cytotechnologist has circled positive cells and struck out negative cells) for feedback on an ongoing basis.

Applying an overlying grid to the printed images has been shown to improve both accuracy of counts and ease of counting.14  Reducing the number of cells per image has also been shown to improve accuracy while decreasing time spent counting.15  Our study photomicrographs included an average of 1765 cells on a single sheet of paper per case. Both the pathologist and cytotechnologist reviewers in our study suggested that this large number of cells per page contributed to the challenge of accurate manual counting in the study materials. This approach was largely done to reduce color printing costs in a study that had multiple reviewers. When implementing the cytotechnologist counting service, we suggest printing and counting multiple images with 500 cells or less per image for each case. We also suggest having the pathologist mark the hot spot on the slide. The cytotechnologist assigned to the quantitative immunohistochemistry service will photograph and print the image for manual counting. The pathologist will have the opportunity to review the manually counted picture prior to sign-out.

In our opinion, applying cytotechnologist skills to this expanded role within surgical pathology would be of great benefit for pathologists and cytotechnologists alike. As cervical screening guidelines change with less frequent screening intervals and more widespread adoption of human papillomavirus testing algorithms, the cervical cytology workload is decreasing. Quantitative immunohistochemistry analysis is a role that can potentially increase the cytotechnologist scope of practice. During a poststudy debriefing session with our 3 cytotechnologists, all expressed enthusiasm for incorporating manual Ki67 counts in their professional practice.

CONCLUSIONS

Our study demonstrates that cytotechnologists can accurately perform manual counts of Ki67 for NET grading, and in turn reduce pathologist workload. This approach has the potential to expand the cytotechnologist's role in the laboratory and improve the accuracy of results used in clinical decision making.

We are very grateful to Carolyn Lohnes, MLT, and Heather MacIntyre, MLT, cytotechnologists at the Queen Elizabeth II Health Sciences Centre, who performed manual Ki67 counts on digital image prints for this study.

References

1
Modlin
IM.
Oberg
K.
Chung
DC.
et al.
Gastroenteropancreatic neuroendocrine tumours
.
Lancet Oncol
.
2008
;
9
(
1
):
61
72
.
2
Fraenkel
M.
Kim
MK.
Faggiano
A.
Valk
GD.
Epidemiology of gastroenteropancreatic neuroendocrine tumours
.
Best Pract Res Clin Gastroenterol
.
2012
;
26
(
6
):
691
703
.
3
Richards-Taylor
S.
Ewings
SM.
Jaynes
E.
et al.
The assessment of Ki-67 as a prognostic marker in neuroendocrine tumours: a systematic review and meta-analysis
.
J Clin Pathol
.
2016
;
69
(
7
):
612
618
.
4
Rindi
G.
Arnold
R.
Bosman
FT.
et al.
Nomenclature and classification of neuroendocrine neoplasms of the digestive system
.
In
:
Bosman
FT.
Carneiro
F.
Hruban
RH.
Theise
ND.
eds
.
WHO Classification of Tumors of the Digestive System. 4th ed
.
Lyon, France
:
International Agency for Research on Cancer (IARC);
2010
:
13
14
.
5
Kroneman
TN.
Voss
JS.
Lohse
CM.
Wu
TT.
Smyrk
TC.
Zhang
L.
Comparison of three Ki-67 index quantification methods and clinical significance in pancreatic neuroendocrine tumors
.
Endocr Pathol
.
2015
;
26
(
3
):
255
262
.
6
Reid
MD.
Bagci
P.
Ohike
N.
et al.
Calculation of the Ki67 index in pancreatic neuroendocrine tumors: a comparative analysis of four counting methodologies
.
Mod Pathol
.
2015
;
28
(
5
):
686
694
.
7
Klimstra
DS.
Modlin
IR.
Adsay
NV.
et al.
Pathology reporting of neuroendocrine tumors: application of the Delphic consensus process to the development of a minimum pathology data set
.
Am J Surg Pathol
.
2010
;
34
(
3
):
300
313
.
8
Tang
LH.
Gonen
M.
Hedvat
C.
Modlin
IM.
Klimstra
DS.
Objective quantification of the Ki67 proliferative index in neuroendocrine tumors of the gastroenteropancreatic system: a comparison of digital image analysis with manual methods
.
Am J Surg Pathol
.
2012
;
36
(
12
):
1761
1770
.
9
Young
HT.
Carr
NJ.
Green
B.
Tilley
C.
Bhargava
V.
Pearce
N.
Accuracy of visual assessments of proliferation indices in gastroenteropancreatic neuroendocrine tumours
.
J Clin Pathol
.
2013
;
66
(
8
):
700
704
.
10
Tuominen
VJ.
Ruotoistenmaki
S.
Viitanen
A.
Jumppanen
M.
Isola
J.
ImmunoRatio: a publicly available web application for quantitative image analysis of estrogen receptor (ER), progesterone receptor (PR), and Ki-67
.
Breast Cancer Res
.
2010
;
12
(
4
):
R56
.
11
Goodell
PP.
Krasinskas
AM.
Davison
JM.
Hartman
DJ.
Comparison of methods for proliferative index analysis for grading pancreatic well-differentiated neuroendocrine tumors
.
Am J Clin Pathol
.
2012
;
137
(
4
):
576
582
.
12
Landis
JR.
Koch
GG.
The measurement of observer agreement for categorical data
.
Biometrics
.
1977
;
33
(
1
):
159
174
.
13
Kane
LE.
Root
RR.
Voss
JS.
et al.
Molecular diagnostics, personalized medicine, and the evolving role of the cytotechnologist: an institutional experience
.
Acta Cytol
.
2012
;
56
(
6
):
678
685
.
14
Voros
A.
Csorgo
E.
Kovari
B.
Lazar
P.
Kelemen
G.
Cserni
G.
The use of digital images improves reproducibility of the Ki-67 labeling index as a proliferation marker in breast cancer
.
Pathol Oncol Res
.
2014
;
20
(
2
):
391
397
.
15
Klapper
W.
Hoster
E.
Determann
O.
et al.
Ki-67 as a prognostic marker in mantle cell lymphoma—consensus guidelines of the pathology panel of the European MCL Network
.
J Hematop
.
2009
;
2
(
2
):
103
111
.

Author notes

The authors have no relevant financial interest in the products or companies described in this article.