Laboratories contemplating either the addition of new molecular tests or modifying methods approved by the Food and Drug Administration for human papillomavirus testing should be aware of a variety of procedural, performance, and regulatory issues surrounding such activity. Diagnostic medical laboratory testing in the United States is regulated by the Centers for Medicare and Medicaid Services, an agency formerly known as the Health Care Finance Administration. The regulatory vehicle of the Centers for Medicare and Medicaid Services is manifested in the Clinical Laboratory Improvement Amendments (CLIA). The CLIA program has put into place specific regulations for laboratory quality control, which includes specific recommendations for method validation. Regulations that must be followed regarding personnel, quality control, quality assurance, method validation, and proficiency testing depend on the complexity category of the individual test. All molecular diagnostic tests, including those for human papillomavirus, are considered high complexity. The Centers for Medicare and Medicaid Services retains the authority to allow private, national accreditation organizations to “deem” that a laboratory is compliant with CLIA '88 requirements. Accreditation organizations, such as the Joint Commission for Accreditation of Hospitals, the Commission on Office Laboratory Accreditation, and the College of American Pathologists (CAP), as well as several state medical laboratory–accrediting agencies, possess the authority to deem laboratories as “CLIA-approved.” The CAP, through its Laboratory Accreditation Program, has promoted standards for laboratory performance and method validation. In general, guidelines set forth in the CAP Laboratory Accreditation Program checklists specify that all clinical laboratory testing must essentially meet those requirements defined for high-complexity testing under CLIA '88, including test validation standards, reportable/reference ranges, performance criteria, and proficiency testing.

The need for effective systems to assure quality and to control the molecular diagnostic laboratory has become apparent during the last decade as these techniques have become commonplace. As routine testing technologies and quality control (QC) procedures have evolved in the routine laboratory, so has the need to maintain compliance within the specialty laboratory. The molecular diagnostic testing laboratory has benefited from the growing pains and establishment of guidelines and procedures in the routine laboratory. Changes in Food and Drug Administration (FDA) requirements establishing the analyte-specific reagent ruling1 have allowed the individual laboratory to conduct in-house validations and to generate diagnostic results from these tests. This advent has led to an enormous increase in the testing capabilities of specialty reference laboratories, which prior to the analyte-specific reagent ruling were limited to reagents for in vitro diagnostic use for clinical diagnostic testing.

All medical laboratory practice (except research) performed on humans for diagnostic purposes in the United States is regulated by the Centers for Medicare and Medicaid Services. The Centers for Medicare and Medicaid Services Division of Laboratory Services, within the Survey and Certification Group under the Center for Medicaid and State Operations, has the responsibility of executing the Clinical Laboratory Improvement Act (CLIA) program. The CLIA program, with an overall objective to ensure quality laboratory testing, has implemented specific regulations for laboratory QC, which include specific recommendations for method validation.2–6 

Regulations that must be followed depend on the complexity category of the individual test. These CLIA complexity categories are waived, moderate complexity, and high complexity. Each of these categories has different regulatory requirements for personnel, QC, quality assurance, method validation, and proficiency testing. All molecular diagnostic tests, including those for human papillomavirus (HPV), are considered to be in the CLIA high-complexity category. As such, all laboratories performing molecular diagnostic tests must ensure that a valid certification of accreditation is in place that covers testing of high complexity. The Clinical Laboratory Improvement Amendments of 1988 (CLIA '88) states:

The laboratory must establish and follow written QC procedures for monitoring and evaluating the quality of the analytical testing process of each method to assure the accuracy and reliability of patient test results and reports.5 

CLIA '88 stipulates that a laboratory that introduces a new procedure for patient testing using a method developed in house, a modification of the manufacturer's test procedure, or a method that has not been cleared by the FDA as meeting CLIA '88 requirements for general QC must, prior to reporting patient test results, perform in-house validation for non–FDA-approved tests.

The Centers for Medicare and Medicaid Services retains the authority to establish and oversee a program that allows national accreditation organizations, such as the College of American Pathologists (CAP) and the Joint Commission for Accreditation of Healthcare Organizations (JCAHO), to “deem” that a laboratory is compliant with CLIA '88 requirements. The Centers for Medicare and Medicaid Services has approved the CAP Laboratory Accreditation Program7 under CLIA '88.

Under JCAHO, the goal of a QC process is to achieve quality in laboratory testing and produce the best possible test results and outcomes. The 2000–2001 Comprehensive Accreditation Manual for Pathology and Clinical Laboratory Services8 states that the goal of improving organizational performance is “for the laboratory to design processes well and systematically monitor, analyze, and improve its services that affect patient health outcomes.”

Activities that allow management to meet these goals include designing processes, monitoring performance through data collection, analyzing current performance, and improving and sustaining performance. The JCAHO provides guidelines for process design for organizational effectiveness and performance improvements through data monitoring. Each specialty and subspecialty (of testing) has a documented QC program. The laboratory must ensure that QC results meet their criteria for acceptability before reporting patient test results. The laboratory's QC system includes daily surveillance of results by appropriate personnel, including taking remedial action for deficiencies identified through QC measures or authorized inspections and documenting such actions.

While accreditation and guidelines for QC are accomplished through CAP and JCAHO, method validation and standardization are accomplished through other organizations. In 1995, the International Organization for Standardization (ISO) established Technical Committee 212 (ISO/TC 212), with the mission of providing quality guidelines for clinical laboratory testing and in vitro diagnostic test systems. NCCLS is the secretariat of ISO/TC 212 and as such manages the development of standards on behalf of the global testing community. NCCLS, a global organization of 2200 organizations in 54 countries, is committed to providing a global forum for development of harmonized standards and guidelines to facilitate safety, best practices, and quality patient care. This objective is accomplished through several organizational goals: (1) achievement of global standardization within medical laboratory testing by developing, adapting, and harmonizing standards that are sensitive and responsive to unique characteristics, customs, cultures, and regulations throughout the world; (2) open communication among partner laboratories by facilitating forums with full participation of all constituencies involved in health care; and (3) dissemination of information and education by raising awareness and developing skills and competence in the health care arena. The outcome of these goals by the NCCLS is to provide worldwide uniformity, consensus guidelines, and standards for health technologies; reduce regional and national variation in regulatory requirements; promote uniformity of testing practices; and facilitate technology transfer. Table 1 lists the appropriate NCCLS guidelines for use in diagnostic medical laboratories performing molecular testing.9–11 

Table 1.

NCCLS Guidelines for Molecular Testing

NCCLS Guidelines for Molecular Testing
NCCLS Guidelines for Molecular Testing

CLIA states that a laboratory introducing a new procedure for patient testing using a method developed in house, a modification of the manufacturer's test procedure, or a method that has not been cleared by the FDA as meeting the CLIA '88 requirements for general QC must, prior to reporting patient test results, verify or establish the performance specifications of the test, establish calibration and control procedures for patient testing as required, and perform ongoing quality assurance. To achieve high-quality results, there are a number of questions that should be addressed in each new validation process. These should include questions regarding the quality process that should be followed, the assay characteristics and specifications, how the assay will be compared (what is the existing method standard for the assay?), technical considerations, and assurances that this new assay meets the required guidelines.

The process control for designing an assay follows 3 major steps: design and verification, validation of the assay, and implementation of the assay with QC procedures (Table 2 and the Figure). The process provides a feedback loop such that all quality improvements can be validated prior to their implementation.

Table 2.

Three Phases of Process Control

Three Phases of Process Control
Three Phases of Process Control

Process control for design, verification, validation, and implementation of the assay

Process control for design, verification, validation, and implementation of the assay

Close modal

As part of the assay validation process and implementation, we must consider the CLIA '88 requirements for protocol validation, as outlined in Table 3. These requirements can be satisfied by referring to the CAP checklist for general laboratory requirements12 and molecular pathology,13 as well as the NCCLS guidelines for interference testing in clinical chemistry14 and assessment of clinical laboratory accuracy.11 

Table 3.

Validation of High-Complexity Molecular Diagnostic Testing Using College of American Pathologists (CAP) Checklists and NCCLS Guidelines*

Validation of High-Complexity Molecular Diagnostic Testing Using College of American Pathologists (CAP) Checklists and NCCLS Guidelines*
Validation of High-Complexity Molecular Diagnostic Testing Using College of American Pathologists (CAP) Checklists and NCCLS Guidelines*

Performance specifications should be established for any new assay. These should take into account comparison with a reference method and any evaluation of bias. The reference method is taken as the gold standard, and performance of any new assay is based on this accepted method. However, the reference method may not necessarily be the best method, in that the new assay may often outperform the accepted method such that it seems inconsequential to perform the comparison. Also, a method must be established for conflict resolution in situations in which disparate results are obtained between the test and reference method. This follow-up may involve resolution of indeterminates with a third method or possibly total rejection of the test method. The overriding fact is that the reference method is the accepted test and must be considered in all cases.

The minimal performance specifications that should be evaluated include accuracy, precision, analytic sensitivity, and analytic specificity, as well as interfering substances, reportable range of results by linearity, and reference range of clinical specimens. Table 4 provides definitions of performance parameters.

Table 4.

Definitions of Performance Measurement Parameters

Definitions of Performance Measurement Parameters
Definitions of Performance Measurement Parameters

High analytic sensitivity can also lead to apparent false positives in situations in which a positive result is seen in patients without the disorder. If the test appears to be “too sensitive,” the reference standard may be “too insensitive.” In other words, the test in question may have a higher positive predictive value than the reference method. For example, such may be the case when comparing the ability of cervical cytology to predict the onset of cervical cancer with the ability of HPV DNA testing to accomplish the same goal.

Process

The process that is followed to measure a new assay system, such as a new collection device for HPV, can be presented in a simplistic form, as shown in Table 5.

Table 5.

Steps in Evaluating a New Assay System for Collection for Human Papillomavirus

Steps in Evaluating a New Assay System for Collection for Human Papillomavirus
Steps in Evaluating a New Assay System for Collection for Human Papillomavirus

Technical Considerations

When considering HPV tests, as well as other molecular diagnostic tests, specimen adequacy or integrity, inhibitory substances, probe cross-reactivity or cross-hybridization effects, false positives, false negatives, and overall predictability of the assay should be evaluated. The types of specimens (Table 6) received can be varied, and false-positive and false-negative rates between different specimen types and methods should be considered in result interpretation.

Table 6.

Specimen Types

Specimen Types
Specimen Types

Cross-contamination and its prevention by the collection process should be a consideration in the processing of the specimen by the clinician or cytology group. False positives and false negatives can result from improper collection and processing of a specimen prior to receipt in the laboratory. Quality control mechanisms can be put in place to monitor the process.

Environmental contamination within the laboratory itself can affect specimen integrity. Contamination can arise from pipette tips and specimen-processing equipment or from a laboratory environment contaminated with polymerase chain reaction–amplified nucleic acid targets (amplicons). The presence of environmental amplicons can contribute to false-positive results if they contaminate negative specimens. Amplicons can be eliminated by incorporation of deoxy-uracil in the polymerase chain reaction and treatment of the starting material with uracil-N-glycosylase (AmpErase, Roche Diagnostics Corporation, Indianapolis, Ind).

Probe cross-reactivity and cross-hybridization can lead to situations in which a patient is referred to a nonnecessary colposcopy procedure. Probe cross-reactivity has been observed with the Hybrid Capture 2 assay (Digene Corporation, Gaithersburg, Md) due to varying degrees of sequence homology, including HPV-13 with the low-risk HPV probe and plasmid pBR322 with both the low- and high-risk HPV probes. Plasmid pBR322 would not be present in a clinical setting, but points to the possibility that a naturally occurring bacterial plasmid in high concentration could potentially cause a cross-hybridization.15 The documented cross-hybridization of the high-risk HPV-42 probe with the low-risk HPV-6 genotype could also lead to the mistaken categorization of a specimen as positive for high-risk HPV.15 Human papillomavirus types 40, 53, and 66 may also cross-hybridize with the high-risk probe; however, there is no evidence to suggest a correlation between infection with these genotypes and development of high-grade disease. Situations of cross-contamination, cross-reactivity, and cross-hybridization all point to the benefit of evaluating an assay for false-positive and false-negative results. The NCCLS guideline GP10-A, “Assessment of the Clinical Accuracy of Laboratory Tests Using Receiver Operating Characteristic (ROC) Plots,” provides standardization calculations for interpretation of false positives and false negatives in the determination of sensitivity and specificity of an assay.11 

The positive and negative predictive values of an assay will be impacted by false-positive and false-negative rates, respectively. These predictive values use the specificity, sensitivity, and pretest probability to calculate a value that indicates whether a test is positive and disease is present or whether a test is negative and no disease is present.

Once a test passes validation and meets performance specifications, it can be implemented into the routine high-complexity laboratory. Validation data for the assay must be retained a minimum of 2 years. Ongoing validation of the procedural accuracy and method comparison should be accomplished in addition to participating in a proficiency testing program for accuracy and reliability at least twice a year. Proficiency testing is performed for all analytes when available, but in the absence of a formal proficiency testing program, the laboratory must develop its own external program (eg, through interlaboratory exchange) or devise some system for internal proficiency testing.

Laboratory accreditation programs7 administered by the CAP examine all aspects of quality improvement in the laboratory and include performance and monitoring of general QC, test methodologies and specifications, reagents, control media, equipment, specimen handling, test reporting, internal performance assessment, external proficiency testing, personnel requirements, safety, document management, and other management practices as included in the inspection process.7,16 Laboratories that meet the accreditation requirements distinguish themselves as quality laboratories.

Accredited laboratories performing molecular diagnostic testing, such as HPV testing, follow guidelines as outlined in the CAP checklists for the general laboratory and for molecular pathology. Documentation and review are required to satisfy personnel competence, provide an audit trail tying the patient's results with the instrument, and QC. A technical supervisor conducts a weekly QC review, and the director or designee performs a monthly review. A review is also required on the next routine shift when there is no on-site supervisor. The QC review must include a mechanism for correlating unexpected test results with other clinical findings.

Quality control is an all-encompassing feature of laboratory endeavor (Table 7). At every stage of a procedure, determination of which QC procedures can be implemented is performed, from the preanalytic stages of specimen collection to result reporting. By being proactive in anticipating what may go wrong within a procedure, preventative measures can be undertaken.

Table 7.

Total Quality Control (QC)

Total Quality Control (QC)
Total Quality Control (QC)

Proficiency testing tells us how well we are performing compared with the rest of the world. This assessment is important when patients' specimens are sent to different laboratories and we assume that the results are comparable. We perform method validation to ensure our new test compares well with our reference method, and in addition we ensure that our test procedures compare with those of other laboratories. A number of proficiency programs enable these comparisons (the CAP's being the most extensive). Internal proficiency testing can be performed using blind samples and repeating patient samples. Documentation and review in all aspects of QC and quality assurance are important aspects of the process to ensure the feedback loop for continuous process improvement.

The ability of a laboratory to reliably turn out high-quality, valid test results is of obvious, paramount importance. To be effective, all aspects of a laboratory's quality program should be viewed as a continual process that must be given consistent attention. Besides accreditation and licensing required by law, laboratories can demonstrate a strong commitment to quality by voluntarily seeking accreditation by the CAP through its Laboratory Accreditation Program.

62 Federal Register 62259
1997
. (
codified at 21 CFR §809.30
).
Medicare, Medicaid and CLIA programs:.
regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA), 57 Federal Register 7002 (1992)
.
Medicare, Medicaid and CLIA programs:.
regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA) and Clinical Laboratory Improvement Act Program Fee Collection, 58 Federal Register 5215 (1992)
.
Medicare, Medicaid, and CLIA programs:.
clinical laboratory requirements—extension of certain effective dates for clinical laboratory requirements under CLIA, 62 Federal Register 25855 (1992)
.
US Department of Health and Human Services.
Clinical Laboratories Improvement Amendments of 1988, 63 Federal Register 55031 (1988) (codified at 42 CFR § 493)
.
Medicare, Medicaid, and CLIA programs:.
extension of certain effective dates for clinical lab requirements under CLIA1, 1998 HCFA-2024-FC, 63 Federal Register 55031 (1998)
.
College of American Pathologists.
Laboratory Accreditation Program Manual.
Northfield, Ill: College of American Pathologists; 2002
.
The Comprehensive Accreditation Manual for Pathology and Clinical Laboratory Services.
Oakbrook Terrace, Ill: Joint Commission on Accreditation of Healthcare Organizations; 2000
.
NCCLS.
Quantitative Molecular Methods for Infectious Disease Proposed Guideline:.
Development and Use of Methods, QA, Proficiency Testing and Interpretation of Results. Publication MM6-P. Wayne, Pa: NCCLS; 2001
.
NCCLS.
Specialty Collection: Evaluation Protocols.
Publication SC1-L. Wayne, Pa: NCCLS; 2002
.
NCCLS.
Assessment of the Clinical Accuracy of Laboratory Tests Using Receiver Operating Characteristic (ROC) Plots.
Approved guideline GP10-A. Wayne, Pa: NCCLS, 1995; reaffirmed May 2001
.
College of American Pathologists.
Laboratory accreditation checklist.
GEN, Laboratory General. Northfield, Ill: College of American Pathologists; 2002
.
College of American Pathologists.
Laboratory accreditation checklist.
MOL, Molecular Pathology. Northfield, Ill: College of American Pathologists; 2001
.
NCCLS.
Interference Testing in Clinical Chemistry.
Publication EP7-P. Wayne, Pa: NCCLS; 1986
.
HPV DNA Test Hybrid Capture 2 [package insert].
Gaithersburg, Md: Digene Corp; 1997
.
College of American Pathologists.
Standards for Laboratory Accreditation. Northfield, Ill: College of American Pathologists; 1997
.

Presented at the College of American Pathologists Strategic Science Series Conference, HPV Testing: Are You Ready for a New Era in Cervical Cancer Screening?, Rosemont, Ill, September 21–22, 2002.

Author notes

Reprints: Jacqueline M. Seabrook, BSc, 250 E Broadway Ave, Maryville, TN 37804 ([email protected])