Identifying Inconclusive Data in the SARS-CoV-2 Molecular Diagnostic Using Nucleocapsid Phosphoprotein Gene as a Target

Nasopharyngeal, nasal, and oropharyngeal swabs and sputum samples (n = 970) collected during May and June 2020 were obtained after SARS-CoV-2 detection in the Laboratory of Vectors and Infection Disease. Residual samples were de-identified samples and considered nonhuman subjects of the research. These samples were used to test the fitness profile of the US CDC 2019-nCoV_N1 and 2019-nCoV_N2 primer-probe sets as described in the following sections.

Sputum and swabs obtained from patients reporting COVID-19–like symptoms were processed to RT-qPCR SARS-CoV-2 detection. In brief, RNA isolation of samples was performed with commercial kits following the supplier's instructions, such as PureLink Viral RNA/DNA Mini Kit (ThermoScientific), Cellco (Cellco Biotec), and Biogene (Quibasa), and RNAs were resuspended in 60 μL of RNase-free water (GIBCO).

RT-qPCR analysis was performed on QuantStudio 5 Real-Time PCR system (ThermoScientific) using the primer set from 2019-nCOV RUO kit (IDT, Coralville, Iowa). The PCR reaction mixture consisted of TaqMan Fast Virus 1-Step Master Mix (ThermoScientific), 0.75 μL of primers, and 2.5 μL of RNA in a final volume of 10 μL reaction. Cycling conditions were 50°C for 5 minutes and 95°C for 20 seconds, followed by 45 cycles at 95°C for 15 seconds and 58°C for 1 minute. Alternatively, we used Kappa Probe Fast qPCR Master Mix (2X) Kit (Sigma-Aldrich), 0.75 μL of primers, and 2.5 μL of RNA in a final volume of 10 μL reaction. Cycling conditions were 42°C for 5 minutes and 95°C for 3 minutes, followed by 45 cycles at 95°C for 5 seconds and 60°C for 1 minute. RNAse P was used as a sample control. The primers and concentrations used in the experiment were as follows: 500 ηM N1: Forward: 5′-GACCCCAAAATCAGCGAAAT-3′; 500 ηM N1: Reverse: 5′-TCTGGTTACTGCCAGTTGAATCTG-3′; and 125 ηM N1-Probe FAM-ACCCCGCATTACGTTTGGTGGACC-NFQ-MGB; 500 ηM N2: Forward: 5′-TTACAAACATTGGCCGCAAA-3′; 500 ηM N2: Reverse: 5′-GCGCGACATTCCGAAGAA-3′; and 125 ηM N2-Probe FAM-ACAATTTGCCCCCAGCGCTTCAG-NFQ-MGB.¹¹  Specificity of the PCR products of N1 and N2 amplification were confirmed by polyacrylamide gel electrophoresis with silver stain.

Single nucleotide polymorphisms (SNPs) in the NP gene were identified by using available data in the National Center for Biotechnology Information (https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/scov2_snp; accessed September 18, 2021). Next, we measured the absolute and relative frequencies of the SNPs present in N1 and N2 regions or outside these regions in the NP gene of the reference genome of SARS-CoV-2 (NC_045512.2:28274-29533).

Descriptive statistics were performed to determine the relative frequencies for categorical variables, as well as to obtain medians and their respective standard error values for continuous variables. Linear regression was used to compare N1 and N2 linearity profiles between both targets to Ct value in all samples used in this study. A receiver operator characteristic (ROC) curve analysis was performed to assess the sensibility and specificity of Ct value in a subset of samples. All data were analyzed with GraphPad Prism 5.0 (GraphPad Software Inc).

The Research Ethics Committee of Universidade Federal do Oeste da Bahia approved this study in 2020 (license No.: 30629520.6.0000.0008). All clinical investigations were conducted according to the Declaration of Helsinki.

Several tests use only 1 set of primers to detect SARS-CoV-2.²  The efficiency or increment of different targets in the detection of SARS-CoV-2 has been poorly addressed. Herein, we evaluate the fitness of the NP gene targets used in the US CDC test to identify SARS-CoV-2 in 970 nasopharyngeal and oropharyngeal swabs or sputum samples. We verified that the N1 and N2 primer sets displayed similar Ct values for each sample (Figure 1, A and B). To verify fitness in each Ct value range, we quantified the frequencies of results between N1 and N2 primer sets. We observed no differences in Ct value failure to detect viral RNA between the N1 and N2 primer sets (Table 1). In addition, we checked the relationship between patient data, such as demographics and days of symptoms, and Ct values (Table 2). We observed no differences in Ct values between N1 and N2 by sex, age, or day of symptoms (Table 2). These data suggest that only 1 primer set could be used to test patients with SARS-CoV-2 infection, potentially reducing costs during molecular testing without diminishing efficiency in diagnostic accuracy.

The major challenge during presence/absence testing in molecular diagnosis is determining the cutoffs for testing. In this work, using commercial templates of the NP gene, we verified the limit of detection and cutoffs of each N1 and N2 primer set (see Supplemental Figure 1 in the supplemental digital content at https://meridian.allenpress.com/aplm in the March 2022 table of contents). Both NP gene targets were able to detect 5 genome copies per microliter of reaction with Ct value of 34.28 ± 0.6841 with N1 and 34.18 ± 0.5382 with N2. In recent work, mock group usage showed high Ct values for the US CDC N molecular test, highlighting the importance of establishing different cutoffs, proposed by standard protocols that recommend a Ct value of 40.^12,13  In this work, we verified the low reproducibility of high Ct values in patient samples, using N1 and N2 US CDC primer sets (Figure 2, A). Also, we identified a usual amplicon size profile in the gel, using both primer sets in those samples (Figure 2, B through D). Our data suggest that the traditional PCR method and acrylamide gels can alternatively be used in remote locales with poor access to molecular tools. SARS-CoV-2 antigen tests are now available at lower costs than molecular methods and could be used to diagnose COVID-19, but they have lower sensitivity and specificity than RT-qPCR.¹⁴ 

To verify the specificity of the US CDC test, we evaluated samples from the same patient that were collected between 2 and 6 days after the first examination and found that only 11 of 32 samples (34.4%) were negative, 10 of 32 (31.2%) had maintained the Ct value, and another 11 of 32 (34.4%) had a reduced Ct value (Figure 3, A). The ROC curve analysis for patients with Ct values above 34.9 that were doubly positive revealed a low sensitivity (52.4%) and specificity (63.6%) of the test in samples with Ct values above 33 (Figure 3, B). Thus, our data suggest that samples tested with Ct values close to the detection limit have a low predictive value and should not be considered for diagnosis before collecting a new sample and performing a second confirmatory test.

Since failure of RT-qPCR detection is related to mutations in the targets used in molecular diagnostics, we studied the SNP identified in the NP gene (Figure 4, A and B). We observed 96 of 1362 SNPs (7.05%) in the N1 region, compared to 40 of 1362 (2.94%) in N2. However, we identified a small number of mutation sites when compared to the 1226 SNPs described outside the target region in the NP gene (Figure 4, C). To verify the impact of the described SNPs in the molecular diagnostics of COVID-19, we compared sequences in the N1 and N2 regions of the NP gene from different variants of SARS-CoV-2, namely P.1, B.1.1.7, B.1.617.2, and B.1.351, obtained from different continents (Supplemental Figure 2, A through C). Surprisingly, only 1 mutation was found in the N1 region of the NP gene (Supplemental Figure 2, B). The data suggest that the NP gene continues to be a suitable target for diagnosing new variants of SARS-CoV-2. However, more studies will be needed to understand the impact of mutations on the efficiency of the different targets in light of the new variants of the virus.

Performance analysis of SARS-CoV-2 virus detection tests has been carried out worldwide.^12,13  The primer and probe set performance for virus detection was evaluated, but few studies have assessed the sensitivity and specificity in the critical range of detection of the RT-PCR technique.²  Among the most sensitive primer and probe sets are those for the N target available through the US CDC.^6,13  In this study, we evaluated the performance of the 2 sets of primers and probes used by the CDC and showed that both presented the same diagnostic performance, suggesting that only one of the targets could be used in the molecular diagnosis of COVID-19, as with other tests that use only 1 molecular target, thus reducing costs.^1,2  Liu et al¹²  in 2020 evaluated the performance of primer and probe sets from different RT-qPCR diagnostic kits for COVID-19, finding results similar to those of our study.¹²  Here, we assessed the performance of the US CDC primer set on sputum and swab patient samples and noted a similar performance between the N1 and N2 targets for virus detection. Some studies have compared the sensitivity between specimens for the detection of SARS-CoV-2 and identified that saliva samples may have a similar or superior sensitivity to swab samples.^13,15,16 

The RT-qPCR is the gold standard technique for detecting the SARS-CoV-2 virus and it has been used to validate alternative diagnostic methods for COVID-19.^1,2,17  However, data concerning sensitivity and specificity of the technique for the higher Ct ranges are still scarce.²  In this regard, a study evaluating the Ct value of health workers who underwent 2 tests found an increase in the Ct value in an interval of 21 days between examinations.¹⁸  Most commercial diagnostic tests recommend that Ct values below 40 be considered as the cutoff point for a positive diagnosis for coronavirus.^12,13,19  Nevertheless, studies that evaluated the detection limit, as well as our study, have shown a low predictive value of RT-qPCR in samples with Ct values above 35, using the US CDC protocol.^12,13  We found that patients with Ct values above 34, when retested within a brief period, may have a drastically altered test result, suggesting that high Ct values have a low positive predictive value. Owing to the COVID-19 pandemic and the growing need for molecular testing, many laboratories have not been able to assess the efficiency of the tests made available for use and are using Ct values below 40 as indicators of positive diagnoses for SARS-CoV-2. We identified, by repeating 2 or more times the extraction of samples with high Ct value, that these samples have low reproducibility, suggesting the need to perform a new US CDC test for greater diagnostic security. We identified 3 reasons for the low reproducibility of samples in the Ct range above 34 for US CDC primer sets: (1) cross-contamination of the samples during processing; (2) low viral load in the samples due to the final or initial stage of infection; and (3) presence of low viral load close to the detection limit of the technique.²⁰ 

Few cases of reinfection have been reported in different countries around the world.^21–24  Reinfection data report a case of mild infection with low viral load and high Ct value followed by a period without positive serology for SARS-CoV-2 infection and a second infection with high viral load and severe symptoms and followed by positive serology.^21,22  Authors should exercise caution in stating cases of reinfection based on high Ct values in either of the 2 episodes reported from the same patient. In addition, a definitive study on humoral response has demonstrated robust long-term production of neutralizing antibodies against SARS-CoV-2 in patients infected only once.²⁵  Evidence of reinfection should consider those cases with a clear viral load in both episodes and for which cross-contamination between samples during the analyses could be ruled out, which could explain both the high Ct values between the samples and the genetic diversity observed. Then, any case report presenting a viral load below a Ct value of 30 in both episodes of infection should be considered reinfection.^21–24 

This study has potential limitations. A small number of patients were examined twice, since it was difficult to get samples of the same patient during the pandemic when testing was scary to the population. In addition, we did not sequence the patient samples and we could not affirm that those Ct failures are related to mutations in the N1/N2 targets, which are very unlikely, considering that during the pandemic period in which the tests were conducted, no cases of variants had been identified in Brazil. Patients enrolled in this study were in distinct stages of the disease. However, we did not have access to complete patient data to conclude whether the Ct value used in diagnostics displays enough of a cutoff to establish a correlation between clinical forms of COVID-19 diagnosis and viral load. Moreover, we recommend more studies using other molecular targets to detect SARS-CoV-2 to verify if our data can be extrapolated to non-N1/N2 assays.

Our data show that both N1 and N2 sets of probes and primers can be used individually for the diagnosis of COVID-19. In addition, we found that the RT-qPCR technique involving US CDC primers should be used with caution in the diagnosis of patients whenever the Ct values are close to the detection limit established by each laboratory service. We recommend that samples with Ct above the detection limit be re-extracted and reanalyzed. In cases of doubt, the patient sample may be tested again by an alternative qPCR protocol, or a new examination must be performed on the patient before the diagnosis can be released as positive. The data from this study impact the interpretation of future data about the COVID-19 pandemic and the conduct of sample analysis using the NP gene as a target.