Context.—The College of American Pathologists' Interlaboratory Comparison Program in Gynecologic Cytology has seen an increase in enrollment in liquid-based Papanicolaou test challenges with a decrease for conventional Papanicolaou tests. Trichomonas vaginalis can be difficult to identify in all preparation types.
Objectives.—To evaluate 20 years of participant results from the College of American Pathologists Interlaboratory Comparison Program in Gynecologic Cytology for Trichomonas to ascertain whether performance has changed because of the introduction of liquid-based Papanicolaou and proficiency testing.
Design.—Concordance rates for the target diagnosis of Trichomonas vaginalis were evaluated for 167 956 participant responses (1990–2010). A nonlinear mixed model was fit with participant type, preparation type, and a 2-level program year (1990–2005 and 2006–2010) reflecting before and after proficiency testing began. A repeated-measures component allowed modeling of the slide-specific performance to ensure that the overall results were not based on the performance of a few slides.
Results.—Cytotechnologists had higher concordance with the target diagnosis than did pathologists (89.8% [72 992 of 81 319] versus 83.4% [72 271 of 86 637], P < .001) and better performance for each preparation type (P = .003). Concordance initially dropped after the introduction of proficiency testing (P < .001) for conventional and liquid-based (SurePath) preparations by both participant types, followed by quick, parallel improvement.
Conclusions.—Performance is high in the detection of Trichomonas vaginalis in the College of American Pathologists Interlaboratory Comparison Program in Gynecologic Cytology. Liquid-based Papanicolaou and proficiency testing minimally affected participant performance. Cytotechnologists performed better over time and across preparation types than did pathologists, although pathologists showed performance results parallel to that of the cytotechnologists. Awareness of the performance differences by pathologists and cytotechnologists, as well as their difference in proficiency among liquid-based techniques, may help ensure accurate results in clinical practice.
Vaginitis caused by Trichomonas vaginalis affects 3 to 5 million women in the United States each year, with a prevalence of 3% among women of reproductive age.1,2 In addition to causing physical discomfort, such as vaginal itching and burning, dyspareunia, and malodorous discharge, Trichomonas infection can have serious health outcomes because it has been associated with preterm birth and delivery of low birth-weight infants in infected, pregnant women and facilitates the transmission of human immunodeficiency virus.3–6 Wet-mount microscopy and recently approved molecular tests by the US Food and Drug Administration are often preferred methods to make this diagnosis, but this infection is frequently detected by Papanicolaou (Pap) tests during cervical cancer screening. Wiese et al7 conducted a meta-analysis of articles published between 1976 and 1998 on the ability of the Pap test to detect vaginal trichomoniasis and demonstrated a 97% specificity among level I studies (ie, those with at least one properly designed, randomized, controlled trial, as defined by the US Preventive Health Task Force8). They, therefore, concluded that treatment is required following a positive Pap test result for Trichomonads in high-prevalence settings.7 Cytologic evaluation, therefore, is important in the diagnosis of this infection, and accurate identification of Trichomonas remains a common and important responsibility for cytotechnologists and pathologists.
The introduction of liquid-based Pap (LBP) tests has provided new opportunities to improve detection of disease, including infections. Liquid-based Pap tests enrich the epithelial cell component and reduce obscuring blood, inflammation, and cellular debris. Removing those components is important to the Pap test's primary purpose of screening for cervical cancer and its precursors and may help unmask organisms, but that process may also eliminate important clues to recognizing infection and decrease the number of pathologic organisms. Liquid-based Pap tests have also introduced new artifacts that potentially create new interpretative challenges. The fixative used in LBPs can cause shrinkage of the organism and the epithelial cells and may alter diagnostic features, such as those typically associated with reactive cellular changes. These changes may make detection of infection more difficult.
The College of American Pathologists Interlaboratory Comparison Program in Gynecologic Cytology (CAP-PAP) provides a unique opportunity to evaluate whether the use of LBPs has changed the performance of cytotechnologists and pathologists in the identification of Trichomonas infection. Because the many participants practice in many different laboratory settings and have a wide variety of background and experiences, this program provides a window into the performance of cytotechnologists and pathologists nationwide. The longitudinal nature of this program also provides an opportunity to evaluate trends in participants' performance over time, compare performance across the various preparation types, and evaluate the effect of mandatory proficiency testing (PT).
MATERIALS AND METHODS
This analysis evaluated 167 956 Trichomonas vaginalis responses from 3730 slides evaluated by pathologists and cytotechnologists participating in the CAP-PAP program between 1990 and 2010. The CAP-PAP is a quality improvement program that fulfills the College of American Pathologists (CAP) Laboratory Accreditation Program requirement that all laboratories evaluating gynecologic cytology enroll in a glass-slide interlaboratory peer-comparison program. Approximately 60% of participating laboratories are hospital-based, with the remainder being independent laboratories, federal and government laboratories, university laboratories, and others.
The CAP-PAP program consists of 5 Papanicolaou-stained glass slides of cervicovaginal material (Pap tests). Slides used in this program are contributed by participants and have been prepared and stained in the contributing laboratories. Slides are reviewed, for the quality of the technical preparation and for its excellence as an example of the submitted diagnosis, by a cytotechnologist and 3 experienced pathologists who are members of the CAP Cytopathology Resource Committee. Before a slide is accepted into the CAP-PAP program, all 3 pathologist reviewers must agree on the exact target diagnosis in addition to agreeing with the contributing laboratory's diagnosis.
Program participants can request slide sets from 3 different types of Pap test preparations: conventional preparations (direct smears), ThinPrep (Hologic, Bedford, Massachusetts) LBPs, and SurePath (BD, Franklin Lakes, New Jersey) LBPs. Slide sets are mailed 4 times per year. The coded answer sheets have diagnostic menus that use terminology modified from the Bethesda 2001 system.9 Referenced slides are assigned to 1 of 3 series: 000 for unsatisfactory slides; the 100 series for normal, infectious, and reparative conditions; and the 200 series for epithelial cell abnormalities (squamous intraepithelial lesions and carcinoma). Within the 100 series, Trichomonas vaginalis is included as a specific, diagnostic response.
The analysis in this study examines concordance of responses from CAP-PAP participants to the exact reference diagnosis of Trichomonas vaginalis. There were several exclusions for this analysis. Data from 1989 were excluded because of missing data for some of the factors, and the responses from the last mailing in 2005 were excluded because that mailing served as the mock Pap proficiency test. Slides with fewer than 10 responses were also excluded from this analysis.
A nonlinear mixed model was fit with 3 factors: reader type, preparation type, and a 2 time intervals (1990–2005 and 2006–2010), which represent periods before and after the introduction of PT. The interaction terms among these factors were also included in the model. The model included a repeated-measures component to include the slide factor correlation, which allows slide-specific performance to be modeled and ensures the overall results are not based on the performance of a few slides. A significance level of .05 was used for this analysis.
RESULTS
The overall concordance rate to the exact reference interpretation was 86.5% (145 263 of 167 956). Reader type, preparation type, and time interval pertinent to PT were all significantly associated with concordance to the reference diagnosis. Additionally, the interactions among preparation type with PT year and reader type were significantly associated with the concordance rate. Cytotechnologists had a better concordance rate than pathologists did overall and for each type of preparation type. In addition, unlike pathologists, cytotechnologists performed equally well with each type of LBP; both pathologists and cytotechnologists were better with LBP preparations than they were with conventional tests, with the poorest performance by pathologists on SurePath Pap tests. The 3-way interaction term was not significant. These results are provided in Table 1.
Effects of Reader, Preparation Type, and Proficiency Testing (PT) on Concordance for Cytologic Detection of Trichomonas Infection
Table 2 summarizes the participant reference interpretations. Table 3 summarizes the reference interpretations by participant type. Papanicolaou tests with Trichomonas are infrequently misdiagnosed as epithelial abnormalities in the CAP-PAP program. The figure plots the annual concordance rates by reader type and demonstrates improved concordance over time for cytotechnologists; pathologist performance has remained relatively unchanged.
COMMENT
The cytomorphologic features of infection with Trichomonas vaginalis, historically described as including inflammatory “cannonball” squamous cells with perinuclear halos, reactive nuclear changes, and attachment of Trichomonas organisms to squamous cells, are well recognized in LBP tests.10,11 The high level of concordance to the reference diagnosis for all 3 preparation types in this study and for both reader types indicates that practitioners are familiar with these features and are able to apply them accurately. Papanicolaou tests with Trichomonas are infrequently misdiagnosed as epithelial abnormalities (2.4%; 4057 of 167 956) in the CAP-PAP program, pathologists almost twice as likely to make this error as cytotechnologists. Cytotechnologist concordance rates were better than those of pathologists for each type of preparation. Unlike pathologists, cytotechnologists performed equally well with each type of LBP and performed better than pathologists did with conventional tests. These findings, however, do not imply that individual cytotechnologists interpret all preparations equally well. The results reflect performance with the participant's preferred preparation type. Data were not collected to compare individual performance with each preparation type for the few participants who chose slide sets with more than one preparation type.
Previous reports have demonstrated that pathologists and cytotechnologists perform differently on various types of challenges in the CAP-PAP program. Cytotechnologists and pathologists show significantly different performance on challenges for herpes simplex virus, although that occurs more often in PT than it does in the CAP-PAP educational program and may reflect different test-taking strategies.9 Cytotechnologists and pathologists also differ in their performance on low-grade squamous intraepithelial lesion educational challenges, with pathologists more often providing an unsatisfactory, benign, or negative response, all of which are considered a major discrepancy from the reference diagnosis.12
As the figure demonstrates, the trends in performance over time for each reader type parallel each other closely. Cytotechnologists have improved much more than pathologists have over the entire period studied, but the trend appears similar for both reader types. Pathologists have also shown significant improvement (P < .05) between 2006 and 2010, following introduction of PT, although their performance lagged behind that of cytotechnologists. Interestingly, a major dip in performance occurred for both reader types in 2006, when PT was introduced, but that may be reflecting removal of the best slides in the educational slide sets in this diagnostic category for use in the PT program. A similar reason may account for a decline in performance for conventional and SurePath preparations following the introduction of PT. Nationally, the market share of these preparations is smaller, resulting in fewer donations to the program and, ultimately, a more-limited pool of slides; those with field-validated responses are removed from the educational sets and put into the sets for PT.
Limitation of this study include (1) findings derived from an educational slide program and not from routine clinical practice, (2) slide challenges prepared in multiple different laboratories, (3) variable number of Trichomonas organisms per unit slide area, and (4) participants from a variety of different laboratories, including commercial laboratories and hospital-based laboratories in both community and academic settings, with wide ranges in the number of tests performed. Alternatively, the broad source of material and participants can also be considered a strength because the results reflect actual clinical material, real-life practitioners, and the environments in which Pap tests are interpreted.
In summary, performance is high in the detection of Trichomonas vaginalis in gynecologic cytology slide challenges in the CAP-PAP program. The introduction of LBP tests and PT appear to minimally affect participant performance. Differences do exist between cytotechnologists and pathologists, with cytotechnologists performing consistently better over time and across preparations types than do pathologists. Awareness of these differences could be important for laboratories in monitoring performance in clinical practice. These differences also emphasize the importance of a multireviewer team-approach to accurate Pap test interpretation.