The College of American Pathologists (CAP), a laboratory accreditation organization with deemed status under the Clinical Laboratories Improvement Amendments of 1988 administers accreditation checklists. Checklists are used by laboratories to ensure regulatory compliance. Peer-level laboratory professionals audit laboratory records during inspections to assess compliance.
To identify the most frequently cited deficiencies for molecular oncology laboratories undergoing CAP accreditation inspections and describe laboratory improvement opportunities.
The CAP Molecular Oncology Committee (MOC), which is involved in maintaining the Molecular Pathology checklist, reviewed data and inspector comments associated with the most frequently observed citations related to molecular oncology testing from laboratories inspected by the CAP during a 2-year period (2018–2020).
Of 422 molecular oncology laboratories that underwent accreditation inspections, 159 (37.7%) were not cited for any molecular oncology–related deficiencies. For the All Common (COM) and Molecular Pathology checklists, there were 364 and 305 deficiencies, corresponding to compliance rates of 98.8% and 99.6%, respectively. The most frequently cited deficiencies are described. The COM checklist deficiencies were associated most often with the analytic testing phase; the MOL checklist deficiencies were more evenly distributed across the preanalytic, analytic, and postanalytic phases of testing.
Molecular oncology laboratories demonstrated excellent compliance with practices that support high-quality results for patients and the health care providers who use those test results in patient management. This review includes a critical assessment of opportunities for laboratories to improve compliance and molecular oncology testing quality.
The College of American Pathologists (CAP) is a member-based physician organization founded in 1946 comprising approximately 18 000 board-certified pathologists. It serves patients, pathologists, and the public by fostering and advocating best practices in pathology and laboratory medicine. The CAP has been given authority as a laboratory accreditation organization by the Centers for Medicare and Medicaid Services (CMS) under the Clinical Laboratory Improvement Amendments of 1988 (CLIA) program. The CAP's accreditation programs ensure that CAP-accredited laboratories are in compliance with CLIA regulations and also meet the needs of patients, physicians, and other health care providers by adhering to clinical laboratory practice standards in these categories: (1) Director and Personnel, (2) Physical Resources, (3) Quality Management, and (4) Administrative Requirements. In 2020, a total of 7778 laboratories were accredited by the CAP.
The CAP provides oversight to laboratories seeking CAP accreditation with 21 accreditation checklists divided into 3 broad (“All Common,” “Director Assessment,” and “Laboratory General”) and 18 specialty checklists, customized on the basis of testing menus of individual laboratories. Each checklist contains a detailed list of requirements that define practice standards. Checklist requirements are at least as stringent as, and often more than, CLIA regulations. Noncompliance with a checklist requirement is called a deficiency. Table 1 includes a description of the types of requirements found in the All Common (abbreviated COM) and Molecular Pathology (abbreviated MOL) checklists. A complete listing of accreditation checklists is publicly available (https://documents.cap.org/documents/cap-accreditation-checklists.pdf; accessed October 14, 2021).
Molecular pathology can be defined as the clinical laboratory–based investigation of patient specimens' nucleic acids. As a laboratory discipline, it is applied across many areas of medicine, for example, microbiology, virology, genetics, hematopathology, oncology, histocompatibility, and pharmacogenomics. Thus, several inspection checklists can be applied to the accreditation process for molecular pathology laboratories. Molecular oncology testing specifically provides diagnostic, prognostic, and predictive information about a patient's neoplastic disease. Molecular oncology investigation also has utility in therapy selection, monitoring responses to therapy, and disease recurrence. This analysis is offered in support of continuous quality improvement and makes use of data specifically attached to accreditation inspection performance of molecular oncology testing laboratories. An examination of the inspection deficiencies, their importance, critical discussion, and opportunities for improvement is presented.
The goal and intent of the CAP is for the inspection process not to be punitive but rather to be focused on improvement, consultation, collaboration, and bidirectional education between the inspector and the laboratory being inspected. It is in this spirit that the CAP Molecular Oncology Committee (MOC) has reviewed data from a recent 2-year period of accreditation inspections and identified the most frequent citations of molecular oncology testing laboratories.
MATERIALS AND METHODS
Data were reviewed from de-identified inspection records of 422 laboratories participating in the CAP's Accreditation Program, using the MOL and COM checklists, during a 2-year period (May 2018–May 2020). Deficiencies that were molecular oncology related from the 2017 and 2018 checklist editions used during the May 2018 to May 2020 inspection period were tabulated. The determination to focus on specific items related to molecular oncology laboratories was made using expert knowledge by members of the CAP MOC and Accreditation Program. While all checklist requirements are applicable to every clinical laboratory seeking CAP accreditation, the scope of this article is limited to clinical molecular oncology testing.
In that context, the authors are aware that there are some requirements in the Cytogenetics and Anatomic Pathology accreditation checklists that are relevant to the practice of molecular oncology. There was insufficient granularity provided to differentiate constitutional from oncology-based testing in cited deficiencies listed for the Cytogenetics checklist's “Arrays” section. This was also the case for in situ hybridization–based deficiencies assessed by the Cytogenetics and the Anatomic Pathology checklists, where it was difficult to differentiate applicability of the deficiencies noted to in situ hybridization versus nonmolecular methods like immunohistochemistry. Data from these checklists were therefore excluded from this analysis; we acknowledge this as a limitation of this report. Ultimately, the analysis and conclusions presented, while arguably not exhaustive, support the aim of this report: to assess meaningful metrics relevant to molecular oncology testing laboratories, thereby providing opportunities for continuous quality improvement.
Deficiencies from the COM and MOL checklists were tabulated as Phase I or II and categorized as relevant to the preanalytic, analytic, and postanalytic phases of testing. Phase I deficiencies, by definition, are citations for checklist requirements that may compromise the quality of services but do not endanger patients, clients, or personnel. Phase I deficiencies require documentation that describes the corrective action taken. Phase II deficiencies are citations for checklist requirements that may seriously impact the quality of services, endanger patients, clients, or personnel, or impact regulatory compliance. Phase II deficiencies require submission to the CAP of a written response of compliance and supporting documentation before accreditation.1
RESULTS
During the 2 years of this study (May 2018–May 2020), 422 molecular oncology laboratories underwent an accreditation inspection (289 domestic, 133 international). Collectively, these laboratories were responsible for 100 858 accreditation requirements, with 31 650 requirements from the COM checklist and 69 208 from the MOL checklist (Table 2). Of the 422 laboratories, 159 (37.7%) were not cited for any molecular oncology–related deficiencies or had such deficiencies ultimately expunged from the final inspection report. Overall, there were 669 deficiencies across the 2 checklists, corresponding to an overall deficiency rate of 0.7% (99.3% compliant). For the COM and MOL checklists, the deficiency rates were 1.2% (364 of 31 650) and 0.4% (305 of 69 208), respectively.
All Common Checklist
The 11 most common citations from the COM checklist included 1 Phase I deficiency and 10 Phase II deficiencies. Two deficiencies deemed “preanalytic” constitute the most and least frequent on the list reported in this analysis. The most common deficiency, cited in 33 of 422 laboratories (7.8%), is COM.01200 (Activity Menu); a laboratory's current CAP testing menu must accurately reflect the testing performed. The least common of the 11 deficiencies included in Table 3, cited in 3 of 422 laboratories (0.7%), is COM.40450 (Analytical Specificity – Modified FDA-Cleared/Approved and LDTs) stating that for modified US Food and Drug Administration (FDA)–cleared/approved tests or laboratory-developed tests (LDTs), the results of each validation study include a sufficient number of samples to establish the test's analytical specificity. Inspectors found in the 3 cited laboratories that appropriate validation for modification of an FDA-approved test was not performed.
The second most frequently cited COM deficiency and the 7 thereafter (Table 3) are categorized in the “Analytic” phase of testing; many concerned proficiency testing (PT). Thirty of 422 laboratories (7.1%) were cited for lack of expiration dates listed on reagents (COM.30400: Reagent Expiration Date). Twenty-eight of 422 laboratories (6.6%) were cited for COM.10000 (Procedure Manual), the requirement that a complete procedure manual be available in the work area. Of the next 6 deficiencies, 4 are related to PT. COM.01500 (Alternative Performance Assessment) and COM.01700 (PT and Alternative Assessment Result Evaluation) were each cited in 19 of 422 laboratories (4.5%) and address alternative performance assessment. COM.01500 states that laboratories must at least semiannually exercise an alternative performance assessment to determine the reliability of analytical performance for tests for which the CAP does not require PT. An inspector's comment appended to a deficiency for this checklist item stated that documentation of corrective action was lacking. Inspector comments related to deficiencies in COM.01700 stated that the issuance of citations was due to lack of review of PT and alternative performance assessment results.
Checklist requirements COM.01100 (Ungraded PT Challenges; 10 of 422 laboratories, 2.4%) and COM.01300 (PT Participation; 9 of 422 laboratories, 2.1%) are also related to PT. While COM.01100 has been significantly revised since the time of this analysis, the 2017 checklist wording specifically required that a laboratory must have a procedure for assessing its performance on PT challenges that are not graded. Cited laboratories lacked these procedures. Revised checklist wording now requires review of all ungraded PT, including educational challenges. As for deficiencies regarding checklist item COM.01300, cited laboratories did not participate in appropriate CAP-accepted PT for patient testing performed.
Three of the 11 most cited deficiencies in this cohort from the COM checklist were COM.04000 (Written Quality Management Program; 17 of 422 laboratories, 4.0%), COM.04250 (Comparability of Instruments and Methods – Nonwaived Testing; 16 of 422 laboratories, 3.8%), and COM.40850 (LDT and Class I ASR Reporting; 5 of 422 laboratories, 1.2%). The 2017 version of COM.04000 states that the laboratory must have a written quality management (QM) program. The checklist notes further specify that the QM program “must ensure quality throughout the pre-analytic, analytic, and post-analytic (reporting) phases of testing, including patient identification and preparation; specimen collection, identification, preservation, transportation, and processing; and accurate, timely result reporting.” This note also included wording mandating that “the program must be capable of detecting problems in the laboratory's systems and identifying opportunities for system improvement” and that “the laboratory must be able to develop plans of corrective action based on data from its QM system.” Review of inspection notes from documented deficiencies for COM.04000 showed that while some cited laboratories had written QM plans, they lacked sufficient goals, plans for corrective action, and documentation of corrective action(s). The COM checklist item, COM.40850, was the only recurring deficiency categorized as “postanalytic.” This item requires that specific wording be included in reports derived from LDTs. Inspection notes related to laboratories deficient in COM.40850 specifically highlighted lack of the required statement that the test was developed in the laboratory issuing clinical reports.
Molecular Pathology Checklist
Citations from the MOL checklist included 8 Phase I and 3 Phase II deficiencies. The 11 most frequently cited deficiencies from inspection with the MOL checklist showed more test component diversity than those from the COM checklist: 4 preanalytic, 5 analytic, and 2 postanalytic.
As with the COM checklist analysis, the most common deficiency cited from the MOL checklist was “preanalytic.” Deficiency in MOL.31130 was cited in 38 of 422 laboratories (9.0%), using the 2017 checklist. MOL.31130 addresses “Accuracy” and specifically states: “The results of each validation study include a sufficient number of characterized samples to provide a high degree of assurance of the test's accuracy.” Inspection comments document that almost half of these citations (17 of 38) were due to lack of samples to validate accuracy of loss of heterozygosity by microarray testing. Note that in the September 17, 2019, checklist edition, MOL.31130 was merged with other requirements for test method validation relating to analytical precision, reportable range, analytical sensitivity, analytical specificity, and other performance characteristics. Additional specific comments made by inspectors relevant to the citations for MOL.31130 included the following: 11 of 38 (29%) for insufficient validation samples with lack of documentation, for example, lot numbers, reagents, operators, and validation study dates; 7 of 38 (18%) for lack of documentation for validation of the lower limit of detection; 4 of 38 (11%) for lack of a documented reportable range; and 3 of 38 (8%) for inability to prove analytical specificity. Some laboratories received multiple specific comments.
Three additional “preanalytic” deficiencies were cited relatively frequently: MOL.36115 (NGS Analytical Bioinformatics Process Validation; 8 of 422 laboratories, 1.9%), MOL.32395 (Neoplastic Cell Content; 8 of 422 laboratories, 1.9%), and MOL.32440 (Ribonuclease-Free Conditions; 6 of 422 laboratories, 1.4%). Checklist items MOL.36115 and MOL.31130, the latter being the most frequently cited deficiency upon inspection with the MOL checklist, both specifically involve test validation. MOL.36115 requires the laboratory to validate the analytical bioinformatics process (also termed bioinformatics pipeline) and revalidate the entire process and/or confirm the performance of the components of the process as acceptable when modifications are made. Inspector comments regarding these deficiencies cited laboratories for not determining the test performance characteristics per variant type, or for not reporting the overall detection rate and confidence intervals. Also commonly cited, MOL.32395 requires records regarding assessment of neoplastic cell content in paraffin-embedded tumor specimens submitted for testing. The least common preanalytic deficiency, MOL.32440, was associated with citations for lack of policy or records on maintenance of ribonuclease-free conditions for assays detecting RNA or those using an RNA probe.
The second and third most common deficiencies for the molecular checklist were in the “postanalytic” category. MOL.20550 and MOL.36155 were each cited in 4.0% (17 of 422) of laboratories. MOL.20550 specifies that when appropriate, statistics on molecular pathology test results, for example, percentages of normal and abnormal findings, must be maintained and appropriate comparative studies performed. MOL.36155 states that the interpretation and reporting of sequence variants should follow professional organization recommendations and guidelines.
The most common category of citations for inspections using the MOL checklist was for metrics associated with the “analytic” component of the laboratory workflow. Five of the 11 citations (Table 4) were categorized as “analytic.” MOL.36135 (Bioinformatics Process/Pipeline – Updates) requires laboratories to have a procedure for monitoring, recording, and implementing patch-releases, upgrades, and other updates to the bioinformatics pipeline. Eight of 422 laboratories (1.9%) were cited for lack of the required procedure specifying a standardized process for bioinformatics updates. MOL.35850 (NGS Confirmatory Testing) requires laboratories to have a written policy that describes the indications for confirmatory testing of report outputs. Seven of 422 laboratories (1.7%) were cited for lack of compliance with this checklist item and specifically for not having documented quality metric thresholds for when confirmatory testing is required or how the laboratory director decides which orthogonal test or methodology to use for confirmation. Inspector comments noted that simply stating that the metrics observed are “not lower than the lowest metrics seen for past variants” is not considered good laboratory practice. Seven laboratories (1.7%) were cited for checklist item MOL.36125 (NGS Bioinformatics Process/Pipeline – Quality Management Program) for not including review of bioinformatics processes as part of a written QM plan.
The final 2 deficiencies from inspections using the MOL checklist identified in this review, both categorized as “analytic” issues, are MOL.36020 (Wet Bench Process Quality Management Program) and MOL.36035 (Monitoring of Upgrades). Deficiencies were issued for lack of compliance with each item in 5 of 422 laboratories (1.2%). MOL.36020 requires laboratories to follow a written QM program for the next-generation sequencing (NGS) analytical wet bench process. Laboratories issued deficiencies for this item were cited because the written QM program did not include specifics regarding use of controls, metrics, and quality control parameters for testing. MOL.36035 requires a written procedure for monitoring upgrades in the laboratory including upgrades to instruments, sequencing chemistries, or reagents and kits used to generate NGS data. Laboratories cited lacked such written procedures.
DISCUSSION
This evaluation represents a review by the CAP MOC of 2 years of laboratory accreditation inspection data from laboratories performing clinical molecular oncology testing. Overall, molecular oncology laboratories performed very well on CAP accreditation inspections. Laboratories were compliant with 98.8% and 99.6% of molecular oncology–related accreditation requirements in the COM and MOL checklists, respectively. High percentages of compliance with accreditation standards reflect that molecular oncology laboratories are operating in a manner consistent with the generation of high-quality test results used by ordering physicians to manage their patients. Domestic CAP accreditation inspections are unannounced; thus these results also suggest that molecular oncology laboratories are consistently meeting these requirements.
Checklist items COM.01200 and MOL.31130 were the 2 most frequently cited deficiencies observed in this cohort of laboratories. COM.01200 requires laboratories to have an accurate activity menu at all times. Maintaining an accurate activity menu is required to ensure appropriate oversight, and providing a current list of tests to the CAP is an important responsibility of accredited laboratories. It is used to monitor PT performance and report specialty information to the CMS. Failure to report correct test information can result in a nonroutine inspection at the expense of the laboratory. With 7.8% of the laboratories in the study cohort cited for this deficiency, laboratories need to consider improving processes for maintaining activity menus. Accuracy of the activity menu is particularly important, since specialty checklists for individual laboratories are often customized to the tests performed. Laboratories should ensure the person or team responsible for updating the activity menu is knowledgeable about the tests performed in the laboratory. Review of the activity menu at regular intervals, including the interim self-inspection, enhances the ability of laboratories to keep activity menus accurate and current. This review may include comparing the directory of services against the CAP activity menu and resolving any discrepancies. In addition, laboratories should incorporate reminders for activity menu updates into procedures and task lists used for implementation or retirement of a test, instrument, or method. In 2019, the CAP Accreditation Program made significant changes to the Molecular Pathology Master Activity Menus to clarify subdiscipline categorization and activities data. The CAP also enhanced access by allowing laboratories to enter and maintain activity information online, using the Organization Profile link on the CAP Web site.
MOL.31130 was cited owing to inadequate studies performed to validate analytical accuracy for an LDT or modified FDA-cleared/approved test. Most commonly, this deficiency was cited owing to an inadequate number or type of specimens used during validation. For example, laboratories may have only used positive samples in the validation. Similarly, the validation may not have included clinically important variants or variant types. The appropriate number of validation samples is dependent on the test, intended use, and methodology used. While the checklist specifies a minimum sample size of 20 for LDT validation, the CAP recognizes that there are situations where a smaller number of positive samples may be appropriate, such as rare mutations or disorders where it may be difficult for the laboratory to obtain positive specimens, for example, CSF3R mutations in chronic neutrophilic leukemia or NRG1 fusions in lung adenocarcinoma. This requirement was significantly revised in the 2019 edition, based on input from relevant CAP committees integral to the peer review process to provide more guidance to laboratories. Laboratories using fewer samples are now required to record the criteria used to determine the sample size of the study. Laboratories must have valid justification(s) for the decision. Literature references on the topic are available.2–5 Specific guidance from the New York State Department of Health Clinical Laboratory Evaluation Program is also available (https://wadsworth.org/sites/default/files/WebDoc/NextGenSeqONCOGuidelines%20FINAL_2020.pdf; accessed October 8, 2021).
It is noteworthy that COM.40450 on validation of analytical specificity precipitated citations in 0.7% of laboratories in this cohort. Inspector comments noted that deficiencies were due to lack of validation of modified FDA-cleared or approved tests. It is important for laboratories to remember that if any modifications are made to an FDA-cleared or approved test, a full validation must be performed, rather than merely a verification of performance. Examples of modifications include the use of unapproved specimen types, collection media, or modification of the assay's lower limit of detection. The validation summary must be reviewed and approved by the laboratory director or designee, who meets CAP director qualifications, before implementing a test. Records of validation data, summary, and approval must be retained for the life of the test, plus at least 2 additional years or longer, based on national, federal, state, and local laws and regulations. These records are needed for inspection and are often useful when troubleshooting potential problems with a test.
COM.30400 mandates that all reagents must be used within their indicated expiration dates. This requirement, based on both CLIA regulations and International Organization for Standardization (ISO) 15189 standards, can be challenging for laboratories given the number of reagents that may be in use at any one time. Laboratories may consider stronger inventory control practices including software-based solutions that prompt active removal of expired reagents. Such systems not only prevent the use of expired reagents, but also assist with the timely ordering of reagents to help prevent shortages.
COM.10000 stipulates the availability of a current procedure manual. More than 6.0% of laboratories in this cohort were cited for this deficiency. Complete and current procedure manuals must be maintained and followed to ensure the standardization of practices and testing quality. Laboratories cited for this deficiency frequently failed to have a procedure manual available at all sites within the laboratory, which includes the laboratory bench. Many laboratories use high-level, bulleted to-do lists, job aids, or abbreviated checklists.6 Uncontrolled aids can be useful but are not a substitute for a fully accessible procedure manual. These aids can also pose risks for version control when the procedure manual is changed. COM.10000 can also be cited when the practice in the laboratory does not match the approved procedure. The best way to avoid these problems is to ensure the laboratory has an effective document control system. Electronic document control systems eliminate reliance on paper copies of procedures and facilitate compliance by removing the need to ensure that the correct version of a procedure is printed for laboratory use. It is imperative to remove outdated procedure manuals and job aids that do not reflect the approved, up-to-date procedure. This is true regardless of whether a laboratory uses an electronic document control system.
PT is a hallmark of best practices in pathology and laboratory medicine. Based on a laboratory's activity menu, CAP-accredited laboratories must enroll and participate in CAP-accepted PT. For tests in which the CAP does not require participation in CAP-accepted PT, laboratories must perform alternative performance assessment to ensure the ongoing quality of testing. In this study, 2.1% of laboratories failed to enroll and participate in a CAP-accepted PT program as required in COM.01300, and 4.5% of laboratories failed to perform alternative performance assessment as required in COM.01500. Laboratories need to carefully evaluate their activity menus to determine which tests require enrollment in CAP-accepted PT and which need alternative performance assessment. Although the rapid pace of change in molecular oncology practice dictates frequent assessment of PT and alternative assessment options, both CAP and other commercial PT programs are available for many tests that require alternative performance assessment. Other acceptable assessment mechanisms may be used at least semiannually, such as split sample testing with referral to other laboratories, split sample analysis with an established in-house method, and the use of assayed materials. Using a spreadsheet or electronic system to track and monitor tests requiring alternative performance assessment can improve compliance.
While careful evaluation of PT and alternative performance assessment results is an essential part of QM, 4.5% of laboratories were cited for not evaluating and taking appropriate corrective action for unacceptable results (COM.01700), and 2.4% were cited for failing to evaluate results not graded by the provider (COM.01100). Laboratories are required to address each unacceptable result within a PT challenge, even if the overall performance score is acceptable. Records of review and investigation of any unacceptable result must be thorough and readily available upon request. Similarly, for results that are not graded by the PT provider, the laboratory must record the acceptability of each result and investigation of unacceptable results. This includes PT challenges that were intended to be graded but were not for various reasons, for example, late submission of results, failure to submit results, incorrect completion of submission forms, and lack of grading consensus. Review of ungraded PT challenges can be done by using the evaluation report and participant summary report provided by the CAP to compare the laboratory's results with the performance of most participating laboratories. A significant revision was made to COM.01100 in the 2020 checklist edition to also require assessment of performance for educational PT challenges. Because many of the ungraded challenges are being used to meet alternative performance assessment requirements, it is essential to record and retain follow-up for these events. The CAP provides guidance and resources, including forms and worksheets, for PT and alternative performance assessment on its Web site in the Proficiency Testing/External Quality Assurance Toolbox.
MOL.32395 describes the requirement to record the histologic assessment of neoplastic cell content on a tissue slide submitted for testing; 1.9% of laboratories in this study were cited for this deficiency. This is a critical preanalytic step in molecular testing that can help ensure specimen adequacy, that is, verify that the quantity and percentage of tumor cells are adequate for the assay and identify findings that may impact the accuracy of the test such as excessive melanin or mucin. Identification of inadequate specimens before molecular testing is begun can help guide oncologists earlier in the testing process, thus potentially reducing the time to treatment initiation.7 Neoplastic cellularity assessment can also save laboratory resources that may otherwise be wasted in the analytic phase of testing. The CAP MOC oversees an educational Neoplastic Cellularity PT Survey that supports laboratories with performance and best practices. Data from this survey have been and will continue to be shared in participant summary reports and the literature.8 Sharing these data is part of an increasing focus by CAP on preanalytical variables affecting the molecular integrity of specimens for precision medicine.9
It is not clear from our analysis if the 1.4% of laboratories cited for MOL.32440 (Ribonuclease-Free Conditions) use environments that could cause RNA degradation or if their operating procedures inadequately addressed this requirement. In either case, these deficiencies reinforce the need for procedures and practices that support quality in the service of optimal patient care. Training in best practices and use of commercial products to preserve RNA will help to prevent ribonuclease-mediated degradation.
Seven laboratories in the study cohort (1.7%) were deficient in MOL.35850, which requires a written procedure on when clinical laboratory test results require confirmation, including somatic variants detected during NGS-based interrogation of tumor-derived nucleic acids. This is a critical requirement. All tests have limitations. Thus, there is a need to confirm results because there are sequences in the genome, for example, high GC content and presence of pseudogenes, that are challenging to amplify, and not all variants and classes of genomic alterations may be optimally identified by a laboratory's methodology. Consistent with the CAP's charge to ensure that clinical laboratories seeking CAP accreditation meet the needs of health care providers and patients who use their services, procedures describing confirmatory testing are a vital part of continuous quality improvement and transparency.
Five laboratories (1.2%) were cited for MOL.36020, which requires a written QM program for the NGS analytical wet bench process. As with any other type of laboratory testing, the performance of NGS tests must abide by the laboratory's QM program, especially given the complexity of the NGS-based processes. Laboratories should ensure all essential elements are included in a QM program and appropriately applied to the NGS-based testing. Failure to do so could impact the quality of patient testing. Many laboratories use the 12 quality system essentials from the Clinical and Laboratory Standards Institute and ISO 15189 to help ensure the completeness of their QM program.10 As part of applying a QM program to the NGS wet bench process, controls, metrics, and quality control parameters must be established during validation/verification and defined in the written procedures, including those used for each analytical run and on an ongoing basis, for example, weekly, monthly, quarterly. Metrics and quality control parameters vary between technology platforms and tests. MOL.36020 was changed from a Phase I to a Phase II requirement in the 2019 checklist edition, which underlies its importance to the safety of laboratory personnel and patients. The CAP recognizes the value of a robust QM program and is continuing to revise the requirement to provide more guidance and detail to laboratories.
MOL.36115, MOL.36125, and MOL.36135 relate to the validation, QM, and system updates for bioinformatics pipelines used in NGS. These requirements were cited in more than 2.0% of laboratories. The integration of bioinformatics into clinical laboratories is a relatively new practice in molecular pathology. Coordination of activities relating to test validation, system updates, and QM is necessary to ensure the quality of NGS-generated test results. Some laboratories use a distributive testing model where wet bench and bioinformatics components may be performed in separate laboratories. This can make compliance with accreditation requirements more challenging. The CAP is continuing to develop and update checklist items related to bioinformatics to optimally support laboratories in this important area of NGS testing. Reviewing and implementing recommendations from consensus guidelines that address the clinical use of bioinformatics pipelines can also help laboratories ensure compliance with accreditation requirements in this evolving area of clinical practice.11
Several themes emerged from this analysis including, but not limited to, (1) bidirectional education opportunities provided by the inspection process, (2) continuous quality improvement, (3) laboratory information management, and (4) the need to revisit this analysis with new data in the future to assess changes and trends in the performance of molecular oncology testing in CAP-accredited clinical laboratories. We note that many of the metrics in Tables 3 and 4, particularly MOL.36035, MOL.20550, and MOL.36020 (and, to a lesser extent, MOL.32395 and COM.10000), may be positively impacted by modern electronic information management systems, that is, laboratory information management systems (LIMS). In this context, it is noted that there is a paucity of molecular pathology–specific LIMS products and this presents a potential opportunity to improve quality through such systems.12
One limitation of this study is that it did not assess whether any of the deficiencies highlighted were recurrent, based on prior on-site inspections. Recurrence of a deficiency on biennial inspections indicates a laboratory's failure to adequately address a deficiency or that a deficiency recurred after it had been resolved. In either scenario, recurring deficiencies pose a threat to laboratory quality and are closely tracked by the CAP. Information on previously cited deficiencies is shared with inspectors as an area of focus during on-site inspections. The CAP Accreditation Program carefully evaluates responses and documentation from laboratories for corrective measures taken whenever a recurrent deficiency is cited for effectiveness to achieve ongoing compliance. Recurring deficiencies may prompt a nonroutine accreditation inspection and may affect a laboratory's accreditation status depending on the nature and seriousness of the deficiency. Future studies on accreditation deficiencies could include more longitudinal analyses to better understand the frequency of recurring deficiencies. We recommend laboratories scrutinize previously cited deficiencies during the performance of self-inspections and during preparation for on-site inspections to ensure deficiencies have been appropriately addressed and that corrective measures have been maintained.
Overall, this study demonstrated that molecular oncology laboratories exhibited excellent performance during CAP accreditation inspections, exhibiting on average 98.8% and 99.6% compliance with molecular oncology–related requirements in the COM and MOL checklists, respectively. These high percentages of compliance with accreditation standards reflect that molecular oncology laboratories are meeting regulatory requirements to generate high-quality results for patients and their treating providers. Given that domestic inspections are unannounced, the results also suggest that laboratories are operating in a state of constant readiness with regard to these markers of laboratory quality. The most common deficiencies identified in this study provide opportunities for laboratories to further improve their quality by focusing on issues most likely to result in a deficiency. The data from this study can help laboratories improve their existing excellent performance on accreditation inspections. This study also provides valuable information for CAP inspectors to guide their inspections and help ensure they thoroughly assess areas proven to be more problematic for laboratories.
The authors acknowledge the excellent volunteer work of the many laboratory accreditation inspectors whose descriptions of cited deficiencies provided valuable material helpful in the development of this manuscript.
References
Author notes
Fernandes is the American Association for Clinical Chemistry representative and Farkas is the Association for Molecular Pathology representative to the College of American Pathologists Molecular Oncology Committee. All other authors are current or past members of the College of American Pathologists Molecular Oncology Committee or the College of American Pathologists Checklists Committee. Briggs, Wielgos, and Vasalos are employees of the College of American Pathologists.
The authors have no relevant financial interest in the products or companies described in this article.
Competing Interests
The identification of specific products or scientific instrumentation is considered an integral part of the scientific endeavor and does not constitute endorsement or implied endorsement on the part of the authors, Department of Defense (DoD), or any component agency. The views expressed in this article are those of the authors and do not reflect the official policy of the Department of Army/Navy/Air Force, DoD, or US Government.