Quality Assurance in Human Molecular Genetics Testing: Status and Recommendations Open Access

The course of laboratory diagnosis of and screening for human disease has been permanently altered by the advent and rapid development of molecular technology and by the completion of the sequencing of the human genome. Findings from the International Human Genome Project and other advances have expanded the applicability of genetic testing to hundreds of conditions in nearly every area of medicine. The use of genetic testing is expanding from the traditional role of diagnostics and screening for inherited diseases to the prediction, prevention, and treatment of chronic diseases of public health importance, such as heart disease and cancer. Populations may be more (or less) susceptible to disease based on their genetic make-up. The susceptibility to or likelihood of developing disease may vary with unique combinations of genetic variations and environmental exposures. The capacity to provide reliable and reproducible tests that detect disease-associated mutations is therefore paramount to identifying at-risk individuals and populations. The issue of quality assurance (QA) is also highlighted in considering that predictive and susceptibility genetic testing is often performed on asymptomatic persons, and interpretation of results may or may not be supported by other findings, such as family history. Furthermore, molecular genetic tests are often performed only once in the life of an individual and may never be repeated or confirmed. The potential for genetic treatment through techniques such as temporary or permanent gene replacement therapy further expands the role of genetics testing.

A milieu of ethical, legal, and social issues also directly affects laboratory genetic testing practices. For example, the need for better understanding of appropriate genetic testing practices was highlighted in a recent study of physicians' interpretations of patients' relative breast cancer risk based on BRCA 1/2 test results.1 Although unique examples of inappropriate genetic testing practices may seem anecdotal, taken collectively, they suggest that the public seems concerned about genetic testing issues.2 Although the criteria that differentiate genetic testing from other types of clinical testing are currently under discussion, tests that analyze DNA to detect sequence variations are being rapidly developed and applied. The application of these recent technical developments to individuals and at-risk populations underscores the need for accurate and reliable human molecular genetic testing (hMGT) and for nationally accepted genetics testing standards and practices. Genetic heterogeneity, regulatory and patent compliance, changing technology, and a generalized lack of understanding among physicians who order tests have been cited as some of the issues presenting new challenges to clinical laboratories that offer genetic testing.3 

The development of generally acceptable quality control methods and QA standards for hMGT has trailed technology development and the integration of tests into clinical practice. Several professional organizations and some states have developed guidelines and continue to propose recommendations and policy statements that address genetic testing quality.4–10 For example, voluntary standards developed by the National Committee for Clinical Laboratory Standards, the American College of Medical Genetics (ACMG), and the College of American Pathologists have served as a framework for testing practices.5,7 Due to the rapid development of the field, gaps that can affect patient care persist. This is of concern to the Centers for Disease Control and Prevention (CDC), other federal agencies, and professional organizations charged with developing regulatory and voluntary standards to ensure appropriate high-quality genetic testing services.

In 1997, a National Institutes of Health/Department of Energy (NIH/DOE) Task Force, assembled to study issues regarding genetics testing, made several recommendations, including that the Clinical Laboratory Improvement Advisory Committee (CLIAC) study genetic testing issues, make recommendations for ensuring the quality of human genetics laboratory testing, and consider the development of a genetic specialty under CLIA (Clinical Laboratory Improvement Amendments) to address issues not currently covered by the general provisions.11,12 Subsequently, in 1998, CLIAC recommended that the Division of Laboratory Systems (DLS), Public Health Practice Program Office (PHPPO), initiate research to define QA needs in hMGT, the most rapidly growing area of genetic testing. In response, the CDC/PHPPO/DLS initiated studies to assess the status of QA and quality control practices of laboratories performing hMGT.13–15 This work was performed to support the DLS mission of continuous improvement in public health through the improvement in the quality of laboratory testing and practices.

In this report, we summarize deliberations of an expert panel in identifying areas needing further development for QA and performance evaluation (PE) programs as applied to hMGT. Specifically, we sought to (1) identify and characterize the focus of QA and proficiency testing (PT) programs for molecular genetics testing, (2) identify a test or group of tests that could be used to monitor quality in hMGT, and (3) specify general recommendations for QA programs for hMGT laboratories. We report herein the outcome of this project.

The current state of clinical molecular genetic testing was initially evaluated by conducting a literature review, making site visits to laboratories that perform such testing, and reviewing available PT programs.15,16 From these efforts, a detailed agenda was developed for discussion by the expert panel. Agendas and background information were sent to panelists 2 weeks before each of 3 panel meetings, and their comments were invited.

Panelists were chosen on the basis of their expertise, availability, and interest in various aspects of hMGT and knowledge of QA practices. Representatives of several professional organizations and federal agencies directly involved in the QA of hMGT served on various panels. Participation and input were primarily from panelists who represented technical, academic, and professional organizations, whereas panelists associated with federal institutions were present primarily for consultation. Most panelists attended all 3 meetings. Panelists are identified in the Appendix.

Three 2-day panel meetings brought the panelists together in 1999. The overarching focus of the meetings was to address the technical aspects and needs for QA in hMGT. Meetings were conducted in both focus group and general session formats. Conclusions and recommendations primarily addressed technical aspects of preanalytic, analytic, and postanalytic phases of genetics testing that affect quality.

In the first meeting, experts reviewed the spectrum and complexity of QA and PT programs currently offered in the United States. Panelists also defined points in the critical pathway of the 4 major technologies used in hMGT, polymerase chain reaction (PCR), sequencing, fluorescence in situ hybridization (FISH), and Southern blot hybridization, which should be targeted for QA efforts. They also discussed the status of current QA of the critical steps and the amenability and importance for further QA efforts.

During the second meeting, panelists discussed the best way to measure competency of laboratories that perform hMGT and identified areas of research that were needed to resolve current problems in QA of hMGT. Discussions included in-depth consideration of various approaches to method-specific QA. Experimental approaches to providing positive specimens were explored.

In the third expert panel meeting, a participatory visioning approach was used to prioritize research and development needs for improving QA in hMGT. A modified nominal group process was used to reach consensus.17 Expert panelists who participated in this meeting were divided into 3 focus groups, roughly based on diversity of expertise or type of experience within the hMGT field. The expert panelists who participated in this meeting are so designated in the Appendix.

Each focus group was handed a randomized list of 20 potential research and development ideas that had been identified during the previous 2 expert panel meetings as needs for QA in hMGT. Each focus group was instructed to spend several hours deliberating separately on the list of needs, to use a rank-ordering process to prioritize the list, and to report the outcome of this process in a subsequent plenary session. The groups were asked to prioritize the list according to 2 criteria: potential impact on QA in the field and urgency of the need. All panelists were encouraged to participate and contribute ideas.

During the subsequent plenary session, a spokesperson for each group presented the group's prioritized list to the entire panel and explained the rationale for the prioritization. This was done in round-robin style with no particular order of groups. Other members of each focus group were invited by their spokesperson to add comments. During this process the other members of the full expert panel were permitted to ask questions for clarification purposes only. Representatives from CDC and the facilitators answered questions for policy or technical clarification only. After all presentations were completed, the lists of priorities were discussed by the entire panel, and ideas were modified and expanded. The priorities were rank ordered on a new list. After a brief period of further discussion, consensus was reached by the entire panel. Five core recommendations for addressing QA in hMGT were generated. Participants in this process suggested that the CDC work with other federal agencies and professional organizations in supporting efforts to address the recommendations made.

In separate deliberations during the third meeting, panelists were also asked to describe an ideal total QA program for hMGT, including preanalytic and postanalytic testing issues. The results from each group were presented, round-robin style, to the entire panel and discussed and summarized.16 

During the first panel meeting, participants identified that the major factor inhibiting development of comprehensive PE, PT, and QA programs is the lack of positive control samples (samples containing well-defined mutations associated with diseases of public health importance). This also inhibits new test development and evaluation. Laboratories attempt to meet these needs through using residual samples from previous referrals and sharing samples with other laboratories. Problems associated with these approaches include the availability of and access to quality control materials. Laboratories in some institutions may also experience barriers associated with logistical issues, such as informed consent requirements. Current PT and PE programs may use samples from interlaboratory sharing and/or from immortalized cell repositories, but resources are limited.15,16 

The panelists pointed out that marked variation exists in the volume of testing and number of laboratories that offer specific genetics tests, complicating the efficiency of conducting PT programs. In fact, many molecular genetics tests are still in the research stage and are performed by only a few laboratories. The panel concurred that the lack of standardized testing methods has complicated the development of some needed QA and PT programs to address several areas of hMGT. Furthermore, the panel pointed out that variability existed in the mutation panels offered by laboratories. This may be less of a problem today due to the official adoption and more widespread use of ACMG recommendations for the offering of a core panel of mutations in recent years.18,19 Furthermore, the analytical methods also vary among laboratories. Thus, the design of a comprehensive PT program, which adequately compares results among methods, will be complex and expensive to implement.

In the second meeting, the panelists focused on 3 major areas of need for QA in hMGT. In addition to the need for more positive control samples, program development and ancillary support were identified for future research initiatives. For developing positive samples, panelists identified several possible approaches that should be considered. These included the development of immortalized cell lines that contain mutations of interest, genetically engineered cell lines that would mimic natural samples in testing, spiked samples, and ex vivo synthesized DNA constructs. The pros and cons of various approaches were considered. Experts advocated developing multiplex samples, containing more than one mutation of interest, which could be used in creating panels for PT and PE programs. They also suggested more research in method-based approaches, designed to test proficiency within a method, but not necessarily of each genetic test for which that method is used. For example, a general method-based approach may be applicable to hMGT for tests in which DNA sequence analysis is performed. Panelists believed that an overall generic or methods-based approach to QA or quality control would be inadequate to cover all aspects of genetic testing at this time. This was partly due to a recognition that test- and disease-specific issues often preclude such a generalized approach. For instance, PCR conditions are often unique to each assay, and conditions that provide for the efficient amplification of one sequence cannot be assumed to be an appropriate measure for successful amplification of other sequences. Further, preanalytic and postanalytic issues vary among tests. With research, however, useful applications of a methods-based approach could emerge for use by laboratorians who have not previously performed hMGT. This approach could also prove useful in laboratories adding or developing new tests for which no PT or PE program is available. The panelists stressed the need for additional pilot studies in expanding PT and PE programs in hMGT for specific technologies, such as sequencing and for tests infrequently performed.

In the third meeting, panelists were asked to conceptualize the ideal QA program for genetics testing laboratories. Their overall consensus was that the ideal QA program would provide assessment of the total testing process from sample acquisition to interpretation of the test result. Preanalytic QA would focus on standardized requisition forms, the informed consent process, and specimen collection. Analytic QA would address test methods, performance, and factors that affect performance. Postanalytic QA would address evaluation of test result reporting and interpretive practices.

Panelists further discussed specific steps in the 4 major genetic technologies, that is, PCR, Southern blot, sequencing, and FISH, for which testing errors may be critical to patient outcome. Panelists pointed out that errors in test validation and interpretation were as critical (or sometimes more critical) to patient outcome as technological errors. Furthermore, although errors in these preanalytic and postanalytic steps could go undetected in traditional PE and PT programs, an ideal QA program should address all steps in the testing process.

The panelists reached a consensus on prioritizing research needs generated during earlier meetings and ranked these needs according to the potential overall impact on quality of hMGT. In ranking these needs, panelists were asked to consider the critical pathway in various hMGT techniques and to define the ways in which the suggested research would lead to improving critical steps in the testing process. The 5 core recommendations generated through the consensus of the panelists for future QA efforts are detailed herein.

In addition, panelists defined some areas relevant to hMGT needing further study that were outside the general scope of this project. These included assistance to the laboratories in assessing the clinical validity of a genetic test before offering it for patient care, QA for interpreting laboratory results in the context of a diagnosis, and training and education requirements for genetics testing personnel.

The panelists concluded that further research and development are essential for assuring quality in hMGT. Furthermore, implementation of recommendations that result from this project would facilitate voluntary standards and program development efforts of states and professional organizations. The following are panelists' recommendations to the CDC for further research and actions:

1. Further research should be conducted in developing positive samples for QA in hMGT. Research should be directed toward producing novel test materials such as engineered samples and facilitation of the process of donating cells to repositories for immortalization. This research responds to the need for positive samples, many of which are not currently in existence or are not generally available to current genetics laboratories or PT or PE providers. Common resources providing perpetually available and accessible positive samples for all known mutations associated with disease are needed. The positive control materials should mimic natural samples usually received in the laboratory as closely as possible. Pilot studies should be conducted to evaluate newly developed materials.

2. The development of PE programs to supplement those in existence was advised. These programs should complement existing PT programs, such as those provided by the College of American Pathologists, but should not duplicate or compete with existing programs. The development of complementary PE programs, particularly for diseases and/or methods not currently addressed by existing PT programs, should be immediately pursued. Currently, this should include addressing PE of orphan tests and of sequencing technology.

3. The establishment and support of disease-specific, laboratory-oriented consortia to provide a forum for information networking and method validation were recommended. Although this recommendation shares similarities with that of the 1996 NIH/DOE Task Force, which advocated disease-specific consortia, it differs in that the consortia proposed herein would specifically address laboratory quality and test performance issues.

4. The establishment and maintenance of a database focused specifically on laboratory testing issues in genetics testing.

5. The support of training programs for personnel involved in molecular genetics testing at all levels was recommended. Panelists generally advocated support of certification programs for those involved in genetics testing.

The need for improvement in the quality of hMGT is recognized by many professional organizations and individuals working in the field.4,8–10,15,19–22 Professional organizations and government entities are focusing efforts toward specific aspects of QA, both nationally and internationally.5,7,15,19,22–24 

The issue of utmost urgency identified by the panelists was the lack of positive samples for daily quality control in testing laboratories and for PE and PT programs. The primary approaches suggested were the use of immortalized cell lines and alternative test materials developed through research. The use of existing immortalized cell lines would require working with repositories to overcome the barriers, including informed consent issues, referencing and characterization of cell lines, and establishing permanent and adequate sources, perhaps through transformation processes. Several possible approaches were discussed for research directed toward developing alternative positive sample materials, including use of spiked samples, engineered samples, and multiplex test samples. Ideally, panels of multiplexed samples, containing several pertinent mutations of interest, would benefit PT and PE programs by decreasing the total number of samples required for shipping. Such samples would likely need to be disease specific. The ideal samples would simulate actual patient specimens closely enough to be useful in QA of multiple steps in the testing process, such as DNA isolation and testing.

In response to these needs, the CDC has contracted to investigate the feasibility of 2 approaches to address the development and availability of needed positive control materials. One approach seeks to establish a process by which laboratories can donate residual patient blood samples in which mutations of interest have been identified. Lymphocytes from donated samples are stably transformed, validated, and made available for a variety of QA purposes. In collecting these samples, we have also addressed patient confidentiality issues and developed procedures acceptable to a variety of institutions, both academic and commercial, through their respective institutional review boards or scientific advisory committees. To date, 21 cell lines that contain mutations associated with diseases of public health importance have been collected, transformed, and reference tested by at least 5 different laboratories using several methods typically used in clinical genetic testing. Some of the targeted diseases for which representative cell lines have been stably transformed are cystic fibrosis, Huntington disease, fragile X syndrome, connexin-26–associated hearing loss, hemochromatosis, α-thalassemia, and sickle cell/hemoglobin C disease. Cells containing factor V Leiden, methylenetetrohydrofolate reductase, and prothrombin mutations that are associated with thromboembolytic risk have also been collected and transformed. Reference testing results have demonstrated the stability of most of the mutations of interest through 5 generations of culture. The one possible exception is the stability of repeat expansions in fragile X cell lines, which is currently under investigation. The process demonstrates a successful approach to providing positive samples for QA of hMGT.24,25,26 Samples from these established cell lines were reference tested by 5 laboratories that offer clinical testing for the targeted mutations. The laboratories that participated in the reference testing used a variety of detection methods. There was overall agreement of results among laboratories in detecting the targeted mutations.25 Samples from the successfully transformed and referenced cell lines will be pilot-tested by 5 other laboratories in a mock PE program. Participating laboratories will be advised of the diseases to test for but will not be given any information about the mutations in the samples. This effort is being undertaken by contract to Duke University with Coriell Cell Repositories and DynCorp Health Research Services as subcontractors.24 

In a separate sample development project, research is being conducted to find novel ways of creating positive samples using genetic engineering techniques and other approaches. The focus is to create samples that mimic patient specimens in commonly used laboratory genetic tests. This project addresses ways of making multiplexed samples and samples that contain mutations that may be difficult to collect. Some of the resulting products were recently pilot-tested in 5 laboratories using at least 2 different analytic methods.24,27 The pilot-testing phase simulated a PE program. This ongoing effort is being conducted by the University of California at Los Angeles under a contract with the CDC.

The panel recommended developing PE programs to supplement those PT programs that already exist. For example, PT and PE are needed for tests based on sequencing technology. Another area of concern is PE for rare diseases, particularly those for which only a few US laboratories are providing testing. The need to address PE for rare diseases and/or polymorphisms associated with chronic disease risk is likely to elevate with the development of technology such as microarray, which will provide the capability of testing for many mutations simultaneously. A method-specific pilot PE program might be appropriate for this type of testing, but approaches for developing such a program need study. Input and cooperation from existing PT providers and stakeholders are essential in designing this type of program. The panel suggested that developing a shared comprehensive technology and methods database would be helpful, especially to enhance QA of rare tests and for developing molecular tests.

The panel pointed out that in hMGT, perhaps more so than in other clinical testing arenas, test utilization, interpretation, and analytical issues are not clear-cut and, in fact, vary with disease. It follows that QA procedures for the total testing process may ultimately and appropriately vary with the particular test. This magnifies the importance of networking and collaboration in ensuring the quality of genetic testing for the public. The establishment and support of laboratory-oriented consortia that would serve as forums for networking, collaborating, participating in pilot research, and providing experience-based information on performance issues were recommended. In addition, the need for linking disease-specific databases and other Internet resources was discussed. An ideal Web-based resource would provide detailed technical information and information regarding test utility and performance. Such a resource might assist laboratories in implementing new tests and foster communication among laboratories involved in genetic testing. Web-based resources that address laboratory issues exist and are evolving to meet laboratory needs. For instance, GeneTests28 is a resource that provides a laboratory directory and detailed information about genetic testing for specific conditions. Disease-specific databases, such as those listed by the National Center for Biotechnology Information,29 are also available. This latter resource and similar ones are useful in discerning genotype-phenotype correlations but lack information about specific testing practices.

The need for improvement in training and continuing education programs for clinicians, laboratory scientists, and technicians who perform tests was highlighted. Since many testing errors occur in the preanalytic and postanalytic phases of testing, it was suggested that better and improved training programs that address the entire testing process from test ordering to reporting and communication of results would be helpful. Existing training and certification efforts could be better coordinated and supported. Also, new Internet-based training tools and resources should be developed. Based on these recommendations, the CDC has bolstered its training and continuing education for clinicians, laboratory scientists, and technicians. In collaboration with Dartmouth Medical College, the CDC, and the Association of Teachers of Preventive Medicine, an interactive multimedia training program, “Genetics in Clinical Practice: A Team Approach,” has been developed.24 This product was designed mainly to help primary care physicians to raise their awareness about genetic issues they are likely to encounter in their practice. Emphasis is placed on the role of genetic testing. Another CDC effort undertaken in collaboration with the public health community was the development of core competencies for genetics in public health. Competencies refer to skills, knowledge, and attitudes necessary for performing one's job. These competencies are currently being used in the development of educational objectives and curricula for laboratory workers and others.23,24 The National Laboratory Training Network, in a collaborative effort between the CDC and the Association of Public Health Laboratories, sponsored 2 training courses that addresses public health aspects of genetics testing: (1) a classroom course entitled “Genes: The Ultimate Survivors,” and (2) a satellite distance learning course entitled “Genetics for the Public's Health.”24,30 Other training projects will continue (see Web site). The CDC continues to enhance its educational role through a cooperative agreement with Mt. Sinai School of Medicine. The objectives of this collaboration include developing assessments aimed at identifying needs in the public health laboratory community and developing continuing educational programs to address those needs.24 

Seeing the continuous need to focus on the QA of genetic testing in general, the CDC has established the Genetics Workgroup in the Laboratory Practice Evaluation and Genomics Branch within DLS/PHPPO. This group has developed a Web site that contains information relevant to laboratory QA of hMGT and other genetic testing. The monograph from this project may be accessed through the newly established Web site, which also provides a link to the CDC genomics Web site, as well as links to other related sites.

Thus, although the CDC has undertaken many activities to address some of the needs outlined through this project, the scope of work described in the 5 core recommendations of the expert panelists is extensive. Accomplishing these goals will require cooperation and collaboration from many partners in both public and private sectors. Although the specific activities and focus may vary among interested groups, it is generally accepted that efforts directed toward improving the quality of genetic testing will strongly affect US public health.

For an individual or family undergoing genetics testing, the stakes are high. Because of the unique issues involved and the impact on families, mistakes in hMGT and genetics testing in general are intolerable. Although genetics testing may pervade all areas of medical practice, it is expected that there will always be hundreds of genetics tests available to the public that will remain infrequently performed. Therefore, the mission of ensuring quality in hMGT must be the joint effort of everyone involved in the field. Furthermore, nationally accepted standards and QA practices for laboratories that perform genetics testing are warranted.

This project was supported by the CDC, Atlanta, Ga, through a contract (200-98-0011) with DynCorp Health Research Services. The authors thank Richard Keenlyside, MD, of the CDC for scientific efforts in this project and Pam Robinson of the CDC for help in manuscript preparation.

Steven Anderson, PhD, Laboratory Corporation of America (LabCorp), Research Triangle Park, NC

Hans Andersson, MD, Tulane University Medical School, New Orleans, La

Jaya Bansal, PhD, Cell Technology, Inc, Jessup, Md

Jeffrey Bartlett, PhD, University of North Carolina, Chapel Hill

Louis Elsas, MD, FFACMG, Emory University School of Medicine, Atlanta, Ga

*Richard Erbe, MD, Children's Hospital of Buffalo, Buffalo, NY

Glen Evans, MD, PhD, University of Texas Southwestern Medical Center, Dallas

*Mary Jo Evans, PhD, Children's Hospital of Buffalo, Buffalo, NY

Andrea Gonzalez, PhD, Medical College of Virginia of VCU, Richmond

*Wayne Grody, MD, PhD, UCLA School of Medicine, Los Angeles, Calif

*Suzanne Hart, PhD, Wake Forest University School of Medicine, Winston-Salem, NC

*Robert Johnson, PhD, Coriell Cell Repositories, Camden, NJ

Kenneth Kidd, PhD, Yale University, New Haven, Conn

Robert Lyons, PhD, University of Michigan, Ann Arbor

*Karla Matteson, PhD, University of Tennessee, Knoxville

*Roger McLendon, MD, Duke University Medical Center, Durham, NC

*James Miller, PhD,† Tulane University Medical School, New Orleans, La

*Catherine O'Connell, PhD, National Institute of Standards & Technology, Gaithersburg, Md

*Vicky Pratt, PhD, FACMG, Laboratory Corporation of America (LabCorp), Research Triangle Park, NC

Theodore Puck, PhD, The Eleanor Roosevelt Institute for Cancer Research, Denver, Colo

*Stuart Schwartz, PhD, Case Western Reserve University School of Medicine, Cleveland, Ohio

*Karen Snow, PhD, Mayo Clinic, Rochester, Minn

*Timothy Stenzel, MD, PhD, Duke University Medical Center, Durham, NC

*Jay Stoerker, PhD, Genzyme Genetics, Framingham, Mass

Petros Tsipouras, MD, University of Connecticut Health Center, Farmington

Linda Wasserman, MD, PhD, University of California-San Diego, La Jolla

*Michael Watson, PhD, Washington University School of Medicine, St Louis, Mo

Ann Willey, PhD, New York State Department of Health, Albany

* These panelists attended the third panel meeting.

† Deceased.