Abstract
Background.—Modern digital cameras can acquire images from cytologic slides at sufficient resolution to allow for digital enlargement and scrolling on a video monitor, allowing for the simulation of microscopy using a computer.
Objective.—The purpose of this study was to develop a tool for proficiency testing in cytopathology using multiple digital images of Papanicolaou tests.
Methods.—Nine images were photographed from each of 10 Papanicolaou tests at ×100 optical magnification, 3400 × 2300-pixel resolution, using a light microscope and a digital camera. All images from each case were tiled in a single canvas with Photoshop 4.0 software. Two cytopathologists and 3 cytotechnologists interpreted these “virtual slides” using a computer and graded diagnostic codes (PAP program, College of American Pathologists). Subjects were retested a year later using the glass slides from the same cases and routine microscopy. Both test results, by diagnostic code, were compared with the McNemar test of symmetry.
Results.—The 5 test subjects provided 42 and 50 correct diagnostic codes by “virtual microscopy” and light microscopy, respectively. No significant asymmetry in results obtained by virtual microscopy and light microscopy was encountered with the McNemar test of symmetry. All test answers were correctly classified by selection series, using both virtual microscopy and light microscopy, and the responses would have been graded as 100% by current PAP program scoring guidelines. This suggests that virtual microscopy could be used for proficiency testing purposes.
Conclusions.—A simple virtual microscopy method designed to challenge participants to locate and diagnose cells of interest was effective for the administration of standardized proficiency tests. Virtual microscopy methods that rely on single-plane images to locate and diagnose cells of interest could provide effective proficiency testing tools prior to the development of more computationally intensive systems that represent an entire Papanicolaou test at multiple focal planes.
The federal government has mandated a national proficiency testing program for gynecologic cytology as a requirement included in the Clinical Laboratory Improvement Amendments of 1988 (CLIA '88).1 This proficiency testing program has not yet been implemented; it would consist of submitting pathologists and cytotechnologists to periodic tests using well-standardized glass slides or computerized methods. Implementation of this CLIA '88 mandate on a national level has proven to be a formidable task. The program requires the availability of a large number of well-characterized test glass slides, many inspectors, and complex logistics to handle the administration of the test to thousands of laboratories dispersed in a large geographic area and management of the data collected from thousands of participants.1–19
The College of American Pathologists (CAP) developed the Interlaboratory Comparison Program in Gynecologic Cytopathology (PAP program, College of American Pathologists, Northfield, Ill) in 1989 as a quality improvement initiative that has since become an important element of the voluntary Laboratory Accreditation Program.20 Scoring criteria for this type of proficiency testing program were developed in 1992, establishing 4 responses, including unsatisfactory, normal or benign changes, low-grade squamous epithelial lesion, and high-grade squamous epithelial lesions and carcinoma.20 The PAP program has grown from an initial enrollment of 207 laboratories to more than 1841 laboratories, 4384 pathologists, and 4109 cytotechnologists as of the year 2000. It has been recently expanded to include a mixture of conventional and liquid-based Papanicolaou (Pap) tests (PAPM program). Participants in the PAP and PAPM programs are mailed sets of 5 glass slides of cervicovaginal materials, 4 times a year.21 These reference slides have been previously carefully screened and validated by at least 3 referees who are members of the CAP Cytopathology Resource Committee, and they are field-validated by at least 20 participants before they become acceptable for use in the PAP program. The participants select diagnoses from a coded answer sheet with graded diagnoses that are equivalent to the diagnostic terminology proposed by the Bethesda System. The diagnoses are also categorized into 3 selection series: 000 for unsatisfactory slides; 100 for normal, infectious, and reparative conditions; and 200 for epithelial abnormalities and carcinoma. The scoring criteria used by the PAP and PAPM programs are less strict than those required in the daily practice of gynecologic cytopathology.20–21 The responses from participants are graded based on whether or not they are in the appropriate selection series of each case. Individual responses are ranked according to their distance from the correct answer, and the scoring system requires a minimum 90% passing score for pathologists. Participating laboratories are expected to have a cumulative score of 90% or greater on 20 validated slides in a 2-year cycle to obtain a passing grade.
It would be very cost effective and advantageous to be able to administer the PAP program or a national proficiency testing program for gynecologic cytopathology using digital images and computer-based methods.5,7 Such “virtual microscopy” methods have been recently used, in our laboratory and others, for teaching and telepathology research applications.6,22 However, to our knowledge there have been few attempts to design and validate a practical proficiency testing program for gynecologic pathology using virtual microscopy techniques. We report a study that simulates 2 mailings of the PAP program to cytotechnologists and cytopathologists and compares the diagnostic scores obtained by examination of glass slides from conventional Pap tests by both light microscopy and virtual microscopy.
METHODS
Case Selection
Ten conventional Pap tests were selected from the files of the cytology laboratory of Cedars-Sinai Medical Center to simulate 2 consecutive mailings of the PAP program. The cases were classified according to the diagnostic codes and selection series used by the PAP program (Table 1). They included a variety of diagnoses, including normal (n = 1); Candida (n = 1); follicular cervicitis (n = 1); inflammation/repair changes (n = 1); Trichomonas vaginalis (n = 1); low-grade squamous intraepithelial lesion (n = 1); and high-grade squamous intraepithelial lesions (n = 4), which included 2 cases of moderate dysplasia and 2 of severe dysplasia. Table 2 summarizes the data from all cases by diagnostic code.
Preparation of Virtual Pap Tests
Proficiency testing programs are designed to test (1) the ability of cytotechnologists and cytopathologists to locate cells of diagnostic interest among a background composed of a much larger number of cellular elements, and (2) the ability to correctly classify the potentially abnormal cells. We attempted to simulate the performance of these 2 basic tasks by acquiring 9 images from each of the 10 Pap tests at ×100 optical magnification, 3400 × 2300-pixel resolution, using a Microlumina digital scanner (Leaf Systems, Marlborough, Mass) attached to a light microscope and a Pentium 400 microcomputer.23–25 The images represented a 1 × 1-cm area of each cytologic slide and included 2 to 3 images with the diagnostic cellular elements and 6 to 7 images with normal cellular elements that could be used as “test distracters.” Each of the individual digital images was saved in Joint Photographic Expert Group (JPEG) format at 20:1 compression. The 9 images from each Pap test were pasted into a single canvas, creating a large image tile for each case (“virtual Pap test”), using Photoshop 4.0 software (Adobe Systems, San Jose, Calif) (Figure 1, A).
A, Virtual Papanicolaou test of a patient with low-grade squamous intraepithelial lesion. Nine images from a conventional Papanicolaou test were acquired at ×100 optical magnification using a high-resolution digital scanner (Leaf Systems, Marlborough, Mass). The 9 images have been organized into a single image tile that represents a 1 × 1-cm area of the glass slide. B, One of the images from the central-left image tile shown in Figure 1, A, has been enlarged digitally to identify koilocytic changes characteristic of human papillomavirus. The images from the virtual Papanicolaou test can be viewed with a personal computer using a standard Web browser and inexpensive plug-in software (Prizm, TMS Sequoia, Stillwater, Okla). The latter software allows for digital enlargement and scrolling on a video monitor, simulating microscopy using a computer. Figure 2. A, Virtual Papanicolaou test of a patient with high-grade squamous intraepithelial lesion. B, One of the hyperchromatic crowded groups of cells seen in Figure 2, A, has been magnified digitally to identify high-grade squamous intraepithelial lesion cells. Dysplastic cells exhibit overlapping, enlarged, hyperchromatic nuclei; high nucleus-to-cytoplasm ratio; and indistinct cytoplasm, and they are arranged in tight clusters. Figure 3. A, Virtual Papanicolaou test of another patient with high-grade squamous intraepithelial lesion. B, The central image from Figure 3, A, has been magnified to identify atypical squamous cells with enlarged hyperchromatic nuclei, densely eosinophilic cytoplasm, and high nucleus-to-cytoplasm ratio. Although the quality of the digital images allowed for the identification of these cytologic features, more subtle findings, such as the presence of irregularities of the nuclear membranes, could not be visualized with our proficiency testing system. The system does not provide users with the ability to change focal planes, limiting their diagnostic capabilities. Future proficiency testing systems will overcome this current technical limitation by constructing image tiles that are photographed at different focal planes and stacked along the z-axis. Such virtual microscopy systems have been developed for investigational purposes but have processing power, computer memory, or bandwidth requirements that are not currently cost effective
A, Virtual Papanicolaou test of a patient with low-grade squamous intraepithelial lesion. Nine images from a conventional Papanicolaou test were acquired at ×100 optical magnification using a high-resolution digital scanner (Leaf Systems, Marlborough, Mass). The 9 images have been organized into a single image tile that represents a 1 × 1-cm area of the glass slide. B, One of the images from the central-left image tile shown in Figure 1, A, has been enlarged digitally to identify koilocytic changes characteristic of human papillomavirus. The images from the virtual Papanicolaou test can be viewed with a personal computer using a standard Web browser and inexpensive plug-in software (Prizm, TMS Sequoia, Stillwater, Okla). The latter software allows for digital enlargement and scrolling on a video monitor, simulating microscopy using a computer. Figure 2. A, Virtual Papanicolaou test of a patient with high-grade squamous intraepithelial lesion. B, One of the hyperchromatic crowded groups of cells seen in Figure 2, A, has been magnified digitally to identify high-grade squamous intraepithelial lesion cells. Dysplastic cells exhibit overlapping, enlarged, hyperchromatic nuclei; high nucleus-to-cytoplasm ratio; and indistinct cytoplasm, and they are arranged in tight clusters. Figure 3. A, Virtual Papanicolaou test of another patient with high-grade squamous intraepithelial lesion. B, The central image from Figure 3, A, has been magnified to identify atypical squamous cells with enlarged hyperchromatic nuclei, densely eosinophilic cytoplasm, and high nucleus-to-cytoplasm ratio. Although the quality of the digital images allowed for the identification of these cytologic features, more subtle findings, such as the presence of irregularities of the nuclear membranes, could not be visualized with our proficiency testing system. The system does not provide users with the ability to change focal planes, limiting their diagnostic capabilities. Future proficiency testing systems will overcome this current technical limitation by constructing image tiles that are photographed at different focal planes and stacked along the z-axis. Such virtual microscopy systems have been developed for investigational purposes but have processing power, computer memory, or bandwidth requirements that are not currently cost effective
Examination of the Virtual Pap Tests Using a Desktop Computer
The test subjects were 2 cytopathologists and 3 cytotechnologists who diagnosed each case by selecting the most appropriate code from a replica of the PAP program answer sheet, using the categories listed in Table 1. The participants were not aware of the proportion of the various diagnostic categories in the test set in either testing situation. Image quality was assessed using a subjective and arbitrary 4-grade scale: poor, marginal, good, excellent. Images were viewed using a desktop computer equipped with Windows 98 (Microsoft, Redmond, Wash) operating system and inexpensive plug-in software (Prizm, TMS Sequoia, Stillwater, Okla).22–25 The latter software allows for image viewing on a computer video monitor in a manner that closely resembles light microscopy, including scrolling by using a “hand tool” and changing magnification digitally up to ×8 without visible image degradation (Figure 1, B). The focal planes of the images could not be changed. Similar virtual microscopy methods have been used in our laboratory for telepathology applications.22–25
Examination of the Same Pap Tests by Light Microscopy
Approximately 1 year after the initial test, the 5 test subjects were retested using routine microscopy for the examination of the glass slides from the same Pap tests that were used to create the virtual Pap test. The glass slides were not prescreened prior to the testing situation.
Test Scoring Criteria: Diagnostic Codes and Selection Series
The test subjects were asked to select the most appropriate diagnostic code for each case. The answers were tabulated by diagnostic method (microscopy or virtual microscopy), diagnostic code, and selection series, as listed in Tables 1 and 2. The correct diagnoses were arbitrarily coded as “1” and the incorrect diagnoses as “2”. The results by light microscopy and virtual microscopy were analyzed with 5 separate McNemar tests for symmetry, comparing the diagnostic codes provided by each of the 5 participants by light microscopy against their own diagnostic codes with virtual microscopy (Systat 10, SPSS Science, Chicago, Ill).
RESULTS
The participants were able to complete the test with virtual microscopy within approximately 5 minutes per case, and with light microscopy in approximately 5 to 10 minutes per glass slide (Figure 2, A and B; Figure 3, A and B). Participants judged the image quality of the virtual Pap tests as either good or excellent. The results of the proficiency testing exercise are shown in Table 2. One cytotechnologist provided 8 of 10 correct diagnostic codes by virtual microscopy, and the other 2 identified 7 of 10 correct codes by this method. Problems in the selection of the correct diagnostic codes included the distinction between normal and reactive changes in 4 instances, between low-grade squamous intraepithelial lesion and high-grade squamous intraepithelial lesion in 2 other cases, and between high-grade squamous intraepithelial lesion and carcinoma in 2 others. The 2 cytopathologists provided all correct diagnostic codes using virtual microscopy. All participants provided 100% correct diagnostic codes using light microscopy for examination of glass slides. Analysis with the McNemar test of symmetry showed no significant asymmetry in the diagnostic codes obtained by the 3 cytotechnologists using virtual microscopy and light microscopy (P = .08 for cytotechnologists A and C, and P = .16 for cytotechnologist B). The sample was too small to provide accurate χ2 approximations. A sample of 10 subjects is underpowered to detect a clinically meaningful difference. The cytopathologists obtained identical diagnostic codes by both methods. All participants provided 100% correct selection series codes by both virtual microscopy and light microscopy.
DISCUSSION
An ideal computer-based proficiency testing method for gynecologic cytopathology would allow participants to view digital images representing an entire cytologic glass slide at the same feature resolution currently available with light microscopy.5,26,27 The digital images would be stacked along the 3-dimensional z plane, allowing cytopathologists and cytotechnologists to change focal planes. The system would be fast enough to allow for the completion of the diagnostic tasks in a time span that is competitive with light microscopy. Such proficiency testing methods are feasible with modern computer technology, but they are not cost effective because they have computational speed and memory requirements that are beyond those available in current desktop computers. Indeed, several commercial companies have already developed sophisticated virtual microscopy systems that allow for the acquisition of digital images representing an entire cytologic or histologic glass slide, usually at a single focal plane. The digitized slides are viewed using a proprietary Web-based browser or other software. Bacus Laboratories (Lombard, Ill) pioneered this technology with the BLISS system, a slide scanner that can automatically scan and digitize images from an entire histologic or cytologic glass slide using a robotics microscope equipped with various objectives up to ×63. The digitized images are automatically arranged in a high-resolution image tile that can be viewed with a computer locally or over the Internet. Current educational programs of CAP, such as the Performance Improvement Program and the Cancer Curriculum, use images acquired with the BLISS system. Interscope Technologies (Pittsburgh, Pa) has developed a competing system, Xcellerator, that combines “rapid image capture with comprehensive data storage and browser-based application software designed specifically for a pathologist's workflow.” Both systems capture multiple small regions of a slide using a microscope with high optical magnification and a charge-coupled device camera, followed by a digital “stitching” together of these image tiles to create a large contiguous digital image of an entire slide. More recently, Aperio Technologies (Vista, Calif) has developed ScanScope, a compact scanner that digitizes histologic or cytologic slides with a technology that operates somewhat like a fax machine. This system can digitize an entire slide within a few minutes at a resolution of 54 000 pixels per inch or higher, close to the resolution available with light microscopy, resulting in very large image files. For example, the digitization of a 15 × 15-mm tissue section with ScanScope results in 2.5 gigabytes of data, before data compression. A stack of images arranged along the z-axis would result in data files that would be several times larger than 2.5 gigabytes.
The test subjects were able to diagnose correctly only 42 of 50 cases by virtual microscopy, and their performance was not as good as on glass slides. These results demonstrate that the virtual microscopy method described in our study cannot be used by cytotechnologists for the classification of Pap tests by diagnostic code. However, the participants were able to classify correctly all cases by selection series using virtual microscopy. The PAP program grades the proficiency tests using selection series, and the results provided by routine microscopy and virtual microscopy would have been scored identically using current CAP scoring guidelines. These preliminary results suggest that it is possible to develop a proficiency testing program for gynecologic cytopathology using virtual microscopy methods that test for subjects' ability to locate cytologic elements of interest and categorize them into graded diagnostic categories. These virtual microscopy systems may not require the use of cytologic slides that have been digitized at multiple focal planes. Additional studies of proficiency tests using virtual microscopy are needed, perhaps using any one of the commercially available scanners described earlier, to determine whether our preliminary results are valid in a larger population of test subjects. These studies could evaluate in a controlled manner what proportion of a glass slide is really needed for proficiency testing purposes and whether the ability to change focal planes significantly changes the test scores. The use of liquid-based methods for gynecologic cytopathology may facilitate the adoption of virtual microscopy methods for proficiency testing purposes. This technology allows for the concentration of cells in a 1.3- to 2-cm-diameter circle, providing a cellular sample for digitization that is considerably smaller than that of a conventional Pap test. Moreover, cytologic slides prepared with liquid-based technology are thinner than routine Pap tests, reducing the possibility of interpretation errors caused by depth of focus problems.