Validation of KRAS Testing for Anti-EGFR Therapeutic Decisions for Patients With Metastatic Colorectal Carcinoma

Before examining the specific performance characteristics of the test, a systematic review of the literature8–12 was performed to determine the appropriate use of the test, whether there was sufficient clinical validation to introduce the test into clinical practice, how the assay results would be used (eg, for diagnosis, prognosis, or as predictor of treatment response), and how the results could change patient management.

A review of the literature indicated that the intended use of the KRAS mutation-analysis test was for the detection of point mutations (substitutions) in codons 12 and 13 of the KRAS gene for anti-EGFR treatment decisions in patients with mCRC. KRAS mutations in these codons have been described in approximately 40% of patients with mCRC and are associated with poor response to treatment with anti-EGFR monoclonal antibodies.6 Several studies5–10,13–15 have shown that only patients with mCRC who have tumors carrying wild-type KRAS demonstrate a favorable response to cetuximab and panitumumab treatment, whereas patients with mutations in codons 12 or 13 do not benefit from these therapies. The only apparent exception to this is the KRAS p.G13D mutation, which has recently been associated with longer overall and progression-free survival among patients with chemotherapy-refractory colorectal cancer treated with cetuximab.16 These data, while intriguing, require further verification. KRAS mutation status is, therefore, a molecular marker for predicting patient response to treatment with anti-EGFR antibodies, and only patients without the KRAS mutations should receive these therapies. Recently published guidelines from the National Comprehensive Cancer Network17 and the American Society of Clinical Oncology18 recommend KRAS mutation testing as part of the evaluation of patients with mCRC who are being considered as candidates for anti-EGFR antibody therapy. More recently, product inserts for both cetuximab and panitumumab recommend KRAS mutation testing for patients with mCRC before drug administration.

KRAS mutational analysis is most frequently performed on primary or metastatic formalin-fixed, paraffin-embedded tumor specimens. Therefore, special specimen collection and handling procedures for formalin-fixed, paraffin-embedded tissues and an appropriate test design for this specimen type must be used. Typical collection and handling procedures include cutting 5 sections (4–7 µm each) on uncoated slides (for most molecular techniques, the tissue size should be at least 0.25 cm²), air-drying unstained sections at room temperature, and staining one of the slides with hematoxylin-eosin. A board-certified pathologist circles an area on the hematoxylin-eosin slide that will be used as a template to guide macrodissection or microdissection of the adjacent, unstained slides. Microdissection is usually performed either with a phase-contrast microscope or with a laser-capture microdissection instrument. The pathologist should provide an estimate of the percentage of neoplastic cells in the area or areas that will be used for DNA extraction, and that percentage should exceed the established limit of detection (LOD) of the assay. For example, if the assay requires that at least 40% of the cells in the area used for KRAS analysis be tumor, then the area that is circled should have more than 40% tumor cells. The minimum percentage of neoplastic cells required will be dependent on the testing technology being used. Tests with higher analytical sensitivity (ie, a lower LOD) will not require as high a tumor percentage. All slides are stored at room temperature and should be sent to the testing laboratory within 5 to 7 days of cutting. Longer storage may lead to decreased quality of isolated DNA.

In this study, the performance characteristics of 2 methods for the detection of KRAS point mutations—Sanger sequencing and allele-specific PCR—were examined. Sanger sequencing was performed using BigDye Terminator v3.1 chemistry (Applied Biosystems, Foster City, California) on the ABI 3100 Genetic analyzer (Applied Biosystems) using laboratory-developed primers. Primers were designed using Primer 3 software (http://frodo.wi.mit.edu/primer3; accessed June 27, 2011) and a reference sequence (GenBank Accession NM_004985.3). The KRAS PCR primers (forward, K12/13 FF, 5′-GTG GAG TAT TTG ATA GTG TAT TAA C-3′; and reverse, K12/13 RR, 5′-TGT ATC AAA GAA TGG TCC TGC A-3′) were designed to amplify exon 2 of the KRAS gene. To compensate for DNA degradation in formalin-fixed, paraffin-embedded tissue, primers were designed to amplify short PCR products of less than 250 base pairs (bp). Reagent concentrations (primers, deoxyribonucleotide triphosphate, magnesium chloride) and PCR conditions were optimized, so a typical PCR amplification assay used 100 ng of DNA, 0.4 µM of each primer, and 0.5 U Platinum Taq DNA Polymerase (Invitrogen, Carlsbad, California). The reaction mixture was subjected to an initial denaturation of 95°C for 2 minutes, followed by 16 cycles of amplification, consisting of denaturation at 95°C for 30 seconds, annealing at 57°C for 30 seconds (reduce annealing temperature 0.5°C per cycle), then extension at 72°C for 30 seconds in a total volume of 50 µL. After that, a further 20 cycles of amplification, consisting of denaturation at 95°C for 30 seconds, annealing at 49°C for 30 seconds, and extension at 72°C for 30 seconds, were performed. The PCR products were purified using the QIAquick PCR Purification kit (Qiagen, Valencia, California), then sequenced in both sense and antisense directions with the primers listed above using the BigDye Terminator v3.1 Cycle Sequencing kit, according to the manufacturer's instructions. Briefly, the reaction mixture was subjected to an initial denaturation of 96°C for 1 minute, followed by 25 cycles of amplification, consisting of denaturation at 96°C for 10 seconds, annealing at 50°C for 5 seconds, and extension 60°C for 4 minutes in a total volume of 20 µL. The sequencing products were purified using a G-50 Sephadex column (Roche Diagnostics Corporation, Indianapolis, Indiana) and analyzed on an ABI 3100. The sequence electropherograms were analyzed, and each case was classified as positive or negative for the KRAS mutation based on the sequencing results (Figure 1, A through C).

For allele-specific PCR, the DxS KRAS mutation test kit (Qiagen) was used. This assay combines 2 technologies, amplification refractory mutation system and Scorpions detection technology to detect mutations in real-time PCR reactions on an ABI 7900 platform (Applied Biosystems). Scorpions are bifunctional molecules containing a PCR primer covalently linked to a probe. The fluorophore in the probe interacts with a quencher, which reduces fluorescence. If a PCR product is generated during the PCR reaction, the probe will bind to the amplicon, and the fluorophore and the quencher will become separated. This leads to an increase in fluorescence. PCR amplification was performed as recommended by the manufacturer, including one control assay (wild-type exon 4 of KRAS) and 7 mutation assays (G12A, G12D, G12R, G12C, G12S, G12V, and G13D). The Scorpions real-time assay uses the number of PCR cycles to determine the number of target molecules present at the beginning of the reaction. The threshold at which the PCR amplification signal is detected above background fluorescence is called the cycle threshold (Ct). Sample ΔCt values are calculated as the difference between the mutation assay Ct and the control assay Ct. Samples are defined as positive if they give a ΔCt that is less than the 2.5% to approximately 5% positive-control ΔCt value for that assay (Figure 2, A through C).

Qualitative tests typically include appropriate negative and positive controls in each run. As true-positive clinical cases could be near the LOD or cutoff, a low-positive control is always included with each clinical run. Seven KRAS mutation-positive cell lines (SW480, MGH8, SW1116, A549, H157, H358, and DLD1) were used as positive controls. Dilutions of these cells lines (at a mutant allele level of 5%–15%), with the negative cell line CASKI, were used as a low-positive control. For sequencing assays, these controls can be rotated among runs to ensure that assays remain cost effective. For assays using the DxS kit, a mixture of all 7 diluted cell lines can be used in every run as controls.

Validation of the analytical performance characteristics of both Sanger sequencing and the DxS assays for KRAS mutation analysis was performed by assessing accuracy, precision, analytical sensitivity and specificity, reportable range, and reference range.

According to the Clinical Laboratory and Standards Institute document EP09-A2,19 ideally at least 40 patient samples, preferably spread throughout many days and runs, should be compared. We analyzed 36 clinical samples and 20 cell line samples that covered the reportable range (wild type and mutant with different mutation levels) against a comparative method (sequencing) or against reference samples obtained from a laboratory with a validated procedure (Tables 1 through 3).

Initially, 2 sets (sections and curls) of samples from patients with mCRC were tested, using the DxS kit and Sanger sequencing (Table 1). All samples were previously tested for KRAS mutations by different laboratories using their techniques. Of 28 tumors, 16 (57%) were positive by sequencing, and 20 of 28 tumors (71%) were positive using the DxS method, illustrating that the sensitivity of the DxS kit was better (approximately 5% of mutant alleles or 10% of cells with mutation in a background of healthy DNA) than that of sequencing (approximately 15% of alleles or approximately 30% of cells with mutation in a background of healthy DNA). The DxS kit confirmed all mutations detected by sequencing, demonstrating excellent concordance between these techniques. However, this technology identified 4 additional tumor samples positive for mutation, which is explained by the higher analytical sensitivity of the DxS method as compared with Sanger sequencing method. These data were confirmed by a third laboratory that also used the DxS kit. Therefore, there was excellent concordance between reference laboratory results for detection of KRAS mutations and those from the other laboratories, with the few discordant cases attributable to differences in the detection sensitivity of the assays used.

To further standardize the KRAS mutation testing methods and procedures, 6 Canadian laboratories from Alberta, British Columbia, Nova Scotia, Ontario, and Quebec participated in 3 separate sample exchanges using KRAS mutation-positive cell lines and paraffin-embedded tissues.20 All participating laboratories detected the appropriate KRAS mutations from undiluted and diluted positive cell lines. However, 2 laboratories had inconclusive results when interpreting 2 positive samples with 10% to approximately 25% mutant allele levels using sequencing (Table 2). These same 2 laboratories were able to detect these mutations using other more-sensitive test technologies. Completely concordant results were achieved by all laboratories for 4 out of 8 paraffin-embedded tissue samples (50%; Table 3). One inconsistent result was later discovered to be due to a sample mix-up (laboratory 4, samples 31 and 32). Partially discordant results were obtained for 3 out of 8 samples (38%; Table 3) because of the small sample size, poor DNA quality, or low level of the KRAS mutant allele. These data emphasize the need for as much information as possible on the material being tested because many variables can influence test results, including test sensitivity and the LOD of the assay.

The accuracy of the Sanger sequencing and DxS KRAS mutation detection assays were assessed using a negative cell line (CASKI) and 7 KRAS positive cell lines that included the 7 most common mutations assayed (Table 2). In addition, 10 clinical specimens (5 positive for mutations and 5 negative for mutations) from an outside reference laboratory that used a clinically acceptable method were also analyzed (samples 10–19; Table 1). The results of testing these known positive or negative samples were 100% concordant with expected values. The close agreement between the results obtained and expected reflects the accuracy of this assay. Of the 180 patient samples tested by the laboratory, 70 (39%) showed the presence of a codon 12 or 13 KRAS mutation, which is very similar to the cited frequency of mutation in the literature.7 

For qualitative assays, precision refers to the reproducibility of test results. Reproducibility is defined by how well independent test results for the same sample agree with one another. Reproducibility of within-run, between-run, between-technologist, and between-instrument results should be assessed. In this validation study, reproducibility was assessed by using 3 lots of reagents, 2 instruments, and 3 operators during 15 runs. For example, using the DxS kit, 8 samples (7 positive for mutation and 1 negative for mutation) were run 15 times as detailed above during the course of 6 weeks. A 100% concordance was achieved for KRAS mutation detection, at a sensitivity of 2.5% to approximately 5%. No significant run-to-run variability was observed (data not shown).

One definition of analytical sensitivity is the ability to obtain positive results in concordance with positive results obtained by a reference method. We had 16 known-positive specimens verified by either sequencing or by a laboratory with a validated procedure. The analytic sensitivity was calculated using the following formula: Analytic sensitivity  =  [True positives/(True positives + False positives)] × 100%  =  ([16/16] × 100%)  =  100%. The analytical sensitivity of the test method using this definition was thus 100%.

More important, analytical sensitivity refers to the smallest amount of a substance that can be accurately measured in a biological sample as characterized by LOD of the assay. The LOD is defined as the lowest amount of analyte that is statistically (at a 95% confidence level) distinguishable from background or from a negative control. For the KRAS mutation test, the LOD was determined using cell-line dilution analysis (data not shown). The LOD was 10% to 15% of mutant alleles (20%–30% of cells with heterozygous mutation) using sequencing and 2.5% to 5% of mutant alleles (5%–10% of cells with heterozygous mutation) using the DxS mutation kit. Knowing the LOD for the particular KRAS assay methodology that the laboratory uses is critically important because it will determine the lowest tumor percentage that can be present in samples that are used for analysis. This also highlights the need for a pathologist to review the tissue to be tested to identify the percentage of neoplastic cells present in the region of interest.

Analytical specificity may be defined as negative agreement when compared with the results obtained by the reference method. Measures that laboratories can take to ensure high analytical specificity include “BLASTing” (using the Basic Local Alignment Search Tool) their primers against the human genome to make sure that they do not cross-hybridize to other regions in the genome, and PCR optimization experiments to ensure that a specific PCR product is generated. In this validation study, 15 specimens were found to be negative in the validation procedure. All 15 specimens were verified to be negative by another laboratory with a validated procedure. The analytic specificity of the assay was calculated using the following formula: Analytic specificity  =  [True negatives/(True positives + False positives)] × 100%  =  ([15/15] × 100%)  =  100%. The analytical specificity of the assay was thus 100%.

The reportable range is defined as the range of test result values for which the laboratory can establish or verify the accuracy of the instrument or test system measurement response. This definition is more applicable to quantitative assays than it is to qualitative assays, such as the KRAS assay, which is simply reported as positive or negative for mutation.

The reference range is the range of test values expected to occur in 95% of a healthy or reference population. For this test, this parameter would be no KRAS mutation detected.

Clinical sensitivity refers to the ability of a test to detect disease when disease is present, and clinical specificity refers to the ability of the test not to detect disease when disease is absent. KRAS mutation analysis is not used to detect disease but to guide treatment, and thus, the terms clinical sensitivity and specificity are not particularly applicable to this assay. The more relevant clinical question for the KRAS assay is its positive and negative predictive value for determining which patients will respond to therapy based on the KRAS test results. Many studies have now demonstrated that the presence of a KRAS mutation has strong predictive value for determining which patients will not respond to anti-EGFR therapies but that the absence of a KRAS mutation does not have strong predictive value for response to treatment.2–10 

It is not a requirement of the Clinical Laboratory Improvement Amendments of 1988 (CLIA) that laboratories determine the clinical sensitivity, clinical specificity, and positive and negative predictive value for laboratory developed tests. However, it is good clinical practice to have an understanding of these performance characteristics as they apply to the specific assay being used. The College of American Pathologists (CAP) Molecular Pathology Checklist21 (MOL.31590) indicates that if a laboratory is unable to address these test characteristics, it should be able to cite studies that have examined or described them.

Criteria were formulated for reporting results for all possible outcomes, for accepting positive and negative results, for defining when tests were to be repeated, and for reporting results as inconclusive.22 For example, using the DxS assay, we report KRAS mutation as positive for G12V, when the ΔCt value is less than or equal to 8.5; report it as negative, when the ΔCt value is greater than 8.5, and the control Ct is less than 29; and report it as inconclusive when the control Ct is greater than 35 (consistent with DNA degradation), or if the JOE (a fluorochrome whose signal is used as an internal control for the presence of amplifiable DNA conditions) signal does not amplify because of PCR inhibition or other factors (Figure 2, A through C). We report results of Sanger sequencing using both amino acid and nucleotide designations; for example, “a KRAS mutation, NM_004985.3:c.35G>A (p.Gly12Asp) has been detected,” when a mutant peak is detected in addition to a wild-type peak.

Templates for reports that indicate the reason for referral, the methodology used, the test result, and clinical interpretations were generated.22 Report interpretations were crafted to indicate that current data suggest that the efficacy of EGFR-targeted therapies (such as cetuximab and panitumumab) in colon cancer is confined to patients with tumors lacking KRAS mutations (with the possible exception of the KRAS p.G13D mutation). We also note in our reports that not all patients who have wild-type KRAS tumors respond to EGFR-targeted therapies; that a wild-type result does not rule out the presence of a mutation that may be less than the LOD for the assay; that the predictive value of KRAS testing applies to EGFR-targeted monoclonal-based therapies, not to other therapeutic agents; and that studies suggest that metastatic and corresponding primary lesions have discordant results in less than 5% of cases.23,24 

Once these validation, interpretation, and reporting steps were completed, the test was incorporated into the routine workflow of the clinical laboratory. The standard operating procedure includes the principle of the test, the clinical utility and intended use of the assay, the appropriate sample types, the specimen handling and storage requirements, the reagents and controls, the instruments, the assay procedures, the result interpretation, the acceptable reporting format and reference ranges, quality control measures, and appropriate references. Participation in proficiency testing was initiated by enrolling in the CAP Molecular Oncology KRAS proficiency testing survey. Quality control and quality assurance criteria, such as turnaround time, documentation of control failures or test failures, and trends in test volumes and results, were defined and monitored. Finally, improvement of the laboratory's current KRAS assays requires an ongoing process of review of new technology and publications in this area as they become available.