Evaluation of a Surrogate Enzyme-Linked Immunosorbent Assay–Based Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) cPass Neutralization Antibody Detection Assay and Correlation With Immunoglobulin G Commercial Serology Assays

A total of 124 samples collected from 81 individuals confirmed positive for SARS-CoV-2 infection by a Food and Drug Administration–authorized reverse transcription polymerase chain reaction (RT-PCR) assay at ARUP Laboratories (Salt Lake City, Utah) were used to evaluate clinical sensitivity. Samples were collected 0 to 36 days post–PCR testing. One hundred samples collected prior to August 2019, before the beginning of the COVID-19 pandemic, were used to evaluate clinical specificity. Of these, 80 were from adults (age range, 20–68 years), and 20 were from pediatric patients (age range, 2–18 years). Ninety-two samples collected from individuals with respiratory illnesses other than COVID-19 (n = 74) as well as individuals positive for rheumatoid factor (n = 12) and heterophile antibody (n = 6) were used to assess cross-reactivity (analytical specificity). All samples were collected, handled, and de-identified in accordance with an institutional review board–approved protocol.

Immunoglobulin G antibody was evaluated on 3 different platforms following the manufacturer's directions: the Abbott SARS-CoV-2 IgG assay (antigen target: N), performed on the Abbott Architect i2000 (Abbott Laboratories Inc, Abbott Park, Illinois); the Euroimmun Anti-SARS-CoV-2 IgG ELISA Assay (antigen target: S), (Euroimmun US, Mountain Lakes, New Jersey), performed both manually and automated on the Dynex Agility (Dynex Technologies, Chantilly, Virginia); and the Siemens SARS-CoV-2 IgG assay (antigen target: RBD), performed on the ADVIA Centaur XPT (Siemens Medical Solutions, Malvern, Pennsylvania).

The cPass SARS-CoV-2 Neutralization Antibody Detection Kit (GenScript, Piscataway, New Jersey) was performed according to the manufacturer's instructions. Briefly, patient serum was mixed with sample dilution buffer (1:10) and horseradish peroxidase–conjugated recombinant SARS-CoV-2 RBD domain (HRP-RBD) fragments. The preincubation step allowed for the binding of circulating nAbs to the HRP-RBD. After 30 minutes at 37°C, the mixture was added to a capture plate that had been precoated with the ACE2 protein. Any unbound HRP-RBD or HRP-RBD bound to nonneutralization antibodies was bound to the plate whereas the circulating neutralization antibody HRP-RBD complexes remained in the supernatant to be removed during the wash step. After washing, tetramethyl benzidine substrate solution was added, followed by the Stop Solution, which quenched the reaction, turning the well color yellow. The plate was immediately read at 450 nm on a microtiter plate reader. Results were reported as percentage inhibition in comparison with the negative control, with a cutoff of 30% indicating the presence of nAbs.

The PRNT assay was performed at Brigham Young University (Provo, Utah) using the SARS-CoV-2 USA-WA1/2020 isolate (Cat # NR-55281, BEI Resources). First, replication-competent virus was grown in Vero E6 cells before being stored at −80°C. A confluent monolayer of Vero E6 cells were then plated in 12-well plates 24 hours before infection. Patient serum samples were serially diluted in Dulbecco modified Eagle medium for a final concentration of 1:25, 1:100, 1:400, and 1:1600. Once the serum was diluted, 200 plaque-forming units of SARS-CoV-2 virus was added and the serum was incubated for 1 hour at 37°C to allow neutralization to occur. After incubation, the mixture was added to the Vero E6 cells and incubated for 1 hour prior to removing the virus-serum mixture and rinsing the cells with 1 mL 1× phosphate-buffered saline. Top agar with 2× Dulbecco modified Eagle medium was then added to the cells and allowed to solidify. After the top agar solidified, the plate was incubated for 3 days at 37°C. Following incubation, cells were fixed with formaldehyde for 1 hour prior to removing the top agar. Cells were then stained with crystal violet and plaques were counted. Finally, 50% neutralization titers (NT50) were calculated using GraphPad Prism (version 8.4.3).

Statistical analysis was performed using Microsoft Excel, AnalyzeIT, version 5.66 (Microsoft Corp, Redmond, Washington) and GraphPad Prism version 8.4.3 (Graphpad Software Inc, La Jolla, California). Positive and negative cut points were adopted from the manufacturer's package inserts. Sensitivity was calculated using the SARS-CoV-2 RT-PCR as the reference method and specificity was assessed using results from the healthy and other infection groups. The sensitivity and specificity for each of the commercial platforms was evaluated using the nAb values as the reference standard. Overall agreement was determined from the proportion of total positive and negative concordant results. Simple linear regression analysis was conducted to determine the correlation coefficients. The area under the curve, predictive values (at a presumed 5% prevalence), and likelihood ratios were determined via receiver operating curve analysis using the Excel AnalyzeIT software.

Sensitivity was assessed based on testing 124 specimens collected at different time points after a positive COVID-19 diagnosis by RT-PCR. The assay showed an overall sensitivity of 76.6% (95% CI, 68.4%–83.2%) post–diagnosis by RT-PCR. As shown in Table 1, 23 of 37 samples collected less than 7 days after being confirmed positive by RT-PCR and 4 of 36 samples collected between 7 and 14 days tested negative by the cPass assay. For samples collected at more than 14 days post-PCR, the sensitivity was 96.1% (95% CI, 94.9%–97.3%). The clinical specificity, evaluated using 100 samples collected prior to the pandemic, showed 100% specificity (95% CI, 98.0%–100.0%) (Table 1). To evaluate the analytical specificity, samples positive either for other respiratory infections or for some of the common analytical interferences such as heterophile antibodies and rheumatoid factors were tested. Of the 92 samples tested, all were negative using the cPass neutralization antibody assay, demonstrating no cross-reactivity and 100% (95% CI, 98.0%–100.0%) analytical specificity. The overall diagnostic ability of the assay to discriminate the positive and negative nAb presence was analyzed using receiver operating curve analysis. At the 30% cut point established by the manufacturer, receiver operating curve analysis showed an area under the curve of 0.91 (95% CI, 0.87–0.96) with a statistical significance of P < .001 (Supplemental Figure 1; see the supplemental digital content at https://meridian.allenpress.com/aplm in the October 2021 table of contents, containing 1 figure and 2 tables). Estimates of seroprevalence prior to the availability of vaccines vary by location but tended to be around 5%.¹¹  At this prevalence, cPass demonstrated a 100% positive predictive value and 86% negative predictive value that correlated with the high specificity observed. At the manufacturer-derived cut point of 30% inhibition, the assay showed a high positive likelihood ratio of greater than 156.4 and a negative likelihood ratio of 0.24 (Table 1).

A panel of 28 RT-PCR–confirmed COVID-19–positive sera with different levels of SARS-CoV-2 nAb were randomly chosen to compare with the conventional PRNT. Simple linear regression analysis showed a 0.76 coefficient of determination when the cPass neutralization antibody percentage inhibition results were compared against the PRNT50 titers (Figure 1, A). Although the 2 methods exhibited a good correlation quantitatively, the cPass neutralization antibody test was designed and has received EUA approval for qualitative detection of SARS-CoV-2 nAbs. A qualitative comparison of the PRNT50 titers with that of the cPass neutralization antibody results for a positive and negative delineation showed an overall agreement of 97.4% (95% CI, 96.2%–98.5%) (Supplemental Table 1). There was only 1 of 28 samples that was collected at an earlier time point (eighth day post–positive RT-PCR confirmation) that did not corroborate with the PRNT50 results and tested negative using the cPass neutralization antibody assay but tested positive using the PRNT assay (Figure 1, B and C). All of the 10 healthy controls tested negative using the PRNT assay, showing a 100% agreement with the cPass neutralization antibody assay (Figure 1, B and C).

At the manufacturer-derived cut points, the overall qualitative agreement between each of the IgG assays and the cPass neutralization antibody assay was evaluated (Table 2). Because of sample volume limitations, not all samples were tested on each of the commercial assays. Of the 117 PCR-confirmed COVID-19–positive samples evaluated, 83 were positive and 22 were negative (most collected at early time points) on both the Abbott and cPass assay. Among the 191 healthy controls and samples positive for infections other than COVID-19, 188 were negative on both platforms, giving an overall agreement of 95.1% (95% CI, 94.6%–95.7%) (Table 2). Fifteen of 308 samples (6 samples <7 days, 4 samples 7–14 days, and 2 samples >14 days collected post–positive PCR confirmation; 1 healthy control; 2 that were positive for infections other than COVID-19) had discordant results between the cPass neutralization antibody assay and the Abbott IgG assay. When comparing Euroimmun with the cPass assay, of the 115 PCR-confirmed positive samples tested, 84 were positive and 23 were negative (most collected at early time points) on both assays. Of the 187 healthy controls and samples positive for infections other than COVID-19, 179 were negative on both platforms, giving an overall agreement of 94.7% (95% CI, 94.1%–95.3%). Results from 16 of 302 samples (4 samples <7 days, 2 samples 7–14 days, and 2 samples >14 days collected post–positive PCR confirmation; 6 healthy controls; 2 that were positive for infections other than COVID-19) were discordant between the cPass neutralization antibody assay and the Euroimmun IgG assay. The Siemens IgG assay exhibited the best agreement with the cPass neutralization antibody results with only 7 of 293 results (4 samples <7 days, 2 samples, >14 days collected post–positive PCR confirmation; 1 that was positive for infections other than COVID-19) being discordant between the two. Of the 105 PCR-confirmed positive samples tested, 74 were positive and 24 were negative on both the Siemens and cPass assays, and of the 189 healthy controls and samples positive for infections other than COVID-19, 188 were negative (most collected at early time points) on both platforms, giving an overall agreement of 97.6% (97.2%–98.0%) (Table 2).

To evaluate whether the commercial serology assays could be used to predict the SARS-CoV-2 neutralization activity as detected by the cPass assay, sensitivity and specificity were calculated using the qualitative cPass neutralization antibody results as the reference standard. At the manufacturer-derived cut point, Abbott had a sensitivity of 94.3% (95% CI, 93.2%–95.4%) and a specificity of 98.4% (95% CI, 98.0%–98.8%) to predict the neutralization activity. Euroimmun had a sensitivity of 93.3% (95% CI, 94.1%–95.3%) and a specificity of 95.7% (95% CI, 95.1%–96.4%). Siemens exhibited the best diagnostic ability to predict the neutralization activity detected by the cPass assay, with a sensitivity of 97.4% (95% CI, 96.6%–98.2%) and a specificity of 99.5% (95% CI, 99.2%–100.0%) (Table 2).

A similar analysis was performed using the PRNT assay positive and negative (cutoff, NT50 titer ≥20) results as the reference standard. The sensitivity and specificity of Abbott were 88.5% (95% CI, 85.5%–91.4%) and 90.0% (95% CI, 85.6%–99.5%), respectively (Supplemental Table 2). Of the 26 PCR-confirmed positive samples tested, 23 were positive on both Abbott and PRNT assay, and of the 10 healthy controls tested, 9 were negative on both platforms. Four of 36 results were discordant, with an overall agreement of 88.9% (95% CI, 86.5%–91.3%) between the Abbott and PRNT assay. Euroimmun predicted with a sensitivity and specificity of 88.9% (95% CI, 86.1%–91.7%) and 100%, respectively. Euroimmun had 24 of 27 results that were concordant with the PRNT assay in the PCR-positive category, and there was zero discordance among the healthy controls, adding up to an overall agreement of 91.4% (95% CI, 89.3%–96.3%). Siemens predicted with a sensitivity of 96.0% (95% CI, 94.2%–97.8%) and specificity of 100%, respectively. The Siemens and PRNT assays showed an overall agreement of 97.1% (95% CI, 95.9%–98.4%), with only one discordant result in the PCR-positive category (Supplemental Table 2).

To evaluate the quantitative correlation, nAb percentage inhibition values derived from cPass assay or NT50 titers obtained from the PRNT assay were compared with the IgG signal to noise ratio acquired from the commercial SARS-CoV-2 IgG serology assays. The comparisons were conducted simultaneously in the 28 randomly selected RT-PCR–confirmed COVID-19–positive samples using simple linear regression analysis (Figure 2, A through F). All 3 platforms included in our analysis showed a positive correlation with the nAb results. When the IgG results from the commercial assays were compared with NT50 titers from the PRNT assay, Siemens (R² = 0.39) displayed the lowest correlation, followed by Euroimmun (R² = 0.57) and Abbott (R² = 0.43). Comparison of the percentage inhibition values obtained from the cPass assay with the IgG signals from the commercial serology assays showed the highest correlation with Euroimmun (R² = 0.85) and comparable degrees of correlation with Abbott (R² = 0.68) and Siemens (R² = 0.63) (Figure 2, A through F).

The SARS-CoV-2 neutralization activity and IgG response detected using the cPass assay and commercial serology assays were longitudinally monitored in 5 RT-PCR–confirmed SARS-CoV-2 patients for a range of 0 to 36 days per the availability of the results. The antibody response was low within the first week, steadily increased over time, and generally peaked around 14 to 21 days across all platforms. Once elevated, neutralization activity plateaued in all 5 patients, grossly comparable with the IgG response detected by the commercial serology assays. The only exception was patient 166, who demonstrated a downward trend in the IgG response detected by the commercial serology assays after 17 to 29 days (Figure 3, A through D).

In the United States, more than 200 SARS-CoV-2 serologic tests are currently available, of which approximately 50 have obtained EUA.¹²  Neutralization antibodies are the only defined correlate to protective immunity to SARS-COV-2 induced either through natural infection or through vaccination.^13,14  Understanding the correlation between protective immunity and clinical protection is a required next phase in the vaccine regimen. A recent study has evaluated the nAb cutoff titers for 50% protection against detectable SARS-CoV-2 infection using data from 7 vaccines available to date.¹⁵  Despite this, there are only a handful of published serology assays for the detection of SARS-CoV-2 nAbs.^{6,9,10,16–20}  The challenges associated with the maintenance of the cell-based live-virus assays and lack of antibody thresholds correlating nAb antibody titers with the protective immunity have limited the availability of the nAb assays in the clinical labs.²¹  To date, only the cPass assay has received EUA for the detection of nAbs. cPass is a simple, faster, and scalable competitive ELISA-based assay for the detection of SARS-COV-2 nAbs.²² 

In this study we assessed the performance characteristics of cPass neutralization antibody assay by comparing it with the gold standard PRNT assay in both qualitative and quantitative manner. A previous study¹⁶  had also confirmed the accuracy of the cPass assay for differentiating nAb-positive and -negative individuals by comparison with viral neutralization tests. Our data were consistent with the qualitative agreement observed between the cPass and PRNT assays used in this study. In our analysis, we also evaluated the quantitative relationship between the cPass percentage inhibition neutralization values with the PRNT50 titers, and our results demonstrated a good positive correlation. Additionally, unique to this study, we evaluated whether the commercial IgG serology assays can predict the nAb activity with the same level of accuracy as the EUA-approved cPass assay. Although the study described above¹⁶  compared the clinical sensitivity and specificity of the cPass assay with those of several other commercial serology assays, the characteristics were compared with reference to PCR positivity and not in correlation with the nAbs to SARS-CoV-2. Our rigorous qualitative and quantitative comparisons across the commercial platforms targeted against different SARS-CoV-2 antigens confirmed the higher accuracy of the cPass assay for the accurate detection of nAbs.

The coronavirus genome encodes 4 main structural proteins: the S protein, the nucleocapsid (N) protein, the membrane (M) protein, and the envelope (E) protein. The S and N proteins are the main immunogens.²³  The S protein consists of 2 subunits: S1, which contains the RBD and N-terminal domain, and S2. Neutralization antibodies are primarily generated against the S1, S2, and RBD domains of the SARS-CoV-2 S protein,²⁴  and the cPass assay detects nAb activity against the RBD region. We used a laboratory-developed PRNT assay in randomly selected PCR-confirmed samples to compare with the cPass assay. The cPass assay has a manufacturer-established cut point of 30% inhibition rate (neutralization activity) that has been correlated²⁵  to an NT50 of 1:20. At this cut point our data revealed a good positive correlation and an excellent qualitative agreement with the PRNT assay, also with an established NT50 of 1:20 as the cut point. A similar but slightly better correlation between cPass and live-virus nAb results has been reported before,⁸  which could be attributed to the differences in the live-virus assay methodology and the lack of precision common to PRNT assays as a result of the subjective nature of scoring the results.

The sensitivity of the cPass assay for the detection of nAb was the highest at time points more than 14 days post–positive PCR confirmation. Although samples collected at less than 7 days following the PCR test had 23 of 124 positive PCR-confirmed specimens that tested negative, 81 of the 87 positive PCR-confirmed specimens collected at more than 7 days following PCR possessed positive neutralization activity. The cPass assay identified 6 PCR-positive samples as negative: 1 was negative in all 3 platforms, 3 other samples coincided with their negative classification on Siemens and Euroimmun but was positive on Abbott, and the remaining 2 samples were discordant between platforms. We note that the 4 patient specimens that did not have a neutralization activity were collected early in the disease process (7–14 days) when there was probably low to no seroconversion. These sensitivity data for the cPass assay using PCR results as the reference were largely comparable with the sensitivities observed on the commercially available Abbott, Siemens, and Euroimmun IgG assays previously published by our group.²⁶ 

It has been hypothesized that the IgG serology assays targeting the RBD domain are better able to predict the neutralization activity against the SARS-CoV-2 virus.⁹  Our data generally supported this hypothesis. Our data revealed not much difference between Abbott's (N) and Euroimmun's (S1) agreement with the cPass neutralization antibody results; however, the Siemens (RBD) assay exhibited the best agreement and had the highest sensitivity to predict neutralization activity. The plausible reason could be the involvement of the same target antigens (RBD) in both the cPass neutralization antibody and the Siemens IgG assays. It is interesting to note that the Abbott assay, which targets N, exhibited an agreement with the nAb results in a similar manner to that of Euroimmun and Siemens, which target S and RBD components, the major elicitors of the nAbs. This is likely due to the similar kinetics and early seroconversion of anti-N and anti-RBD proteins in response to SARS-COV-2 infection. It has been reported in the literature that the detection of both RBD and the N protein is more sensitive than the detection of S1 or S2, respectively.²⁷  Although RBD is a part of the S1 protein, the differences observed in the anti-S1 and anti-RBD responses may be attributed to the presence of cryptic epitopes within the RBD region of S1.²⁸  Our longitudinal follow-up on select patients revealed an increasing trend in the nAb response for the first 2 weeks post–PCR confirmation, which plateaued eventually. This was consistent with the observations made by other groups^29–31  that reported a variable but increasing nAb response and a plateauing 2 weeks post–symptom onset. Although many have reported the trend and kinetics of the SARS-COV-2 antibody response in the short term, the long-term kinetics and durability of the nAbs are still in question.^32–34 

Neutralization antibodies represent only a subset of antibodies produced and not all binding antibodies have the blocking ability. Currently, manufacturers such as Diasorin and Siemens are moving in the direction of associating the IgG (binding and blocking) positive results detected in their assay with a nAb (blocking) claim.¹²  We calculated the sensitivity and specificity of the commercial IgG serology assays in the prediction of nAb activity using cPass results as reference. Noticeably, the majority of samples that were collected more than 14 days post–PCR confirmation and detected with antibodies by the commercial IgG serology assays had a positive nAb presence (>97% sensitivity). Similarly, the majority of the negative IgG results in the healthy donors acquired from the Abbott and Siemens assays correlated with negative neutralization activity (>99% specificity); Euroimmun had a slightly lower specificity (approximately 93%). Samples with nAb that were discordant with the IgG results largely represented samples that were closer to the cutoff. Overall, IgG serology results showed a high level of association with the presence and absence of nAbs, as detected by the cPass assay.

However, when IgG serology results were correlated with the PRNT assay, the sensitivity for the detection of the positive NT50 titers were largely variable; Siemens exhibited the highest sensitivity (>97%) compared with Abbott and Euroimmun. Furthermore, PRNT50s or percentage inhibition values acquired from the cPass assay in SARS-CoV-2 PCR-confirmed positive patients correlated with the corresponding signal to noise ratio on the Abbott, Euroimmun, and Siemens platforms at varying degrees in a poor to modest manner. Although anti-S1 IgG results from Euroimmun exhibited a poor qualitative agreement with nAb presence, the IgG signals showed the highest quantitative correlation with the nAb results. This is likely due to the linear characteristics of this platform. Overall, IgG signals generated by the serology platforms displayed varying degrees of quantitative correlation with the nAb antibody results. One other critical factor to consider is the differences in the antibody classes detected by these assays. Neutralization antibody assays are isotype independent, and the results are reflective of the combined activity of all nAbs in a sample, whereas the serology assays are IgG specific. This could likely cause differences in the mechanism of detection and in assay signal output relating to the antibody concentrations.

This study is limited by the lack of clinical data to correlate the disease severity with the magnitude and kinetics of the nAb response. Additionally, our study cohort did not include samples that were collected at longer time points from initial COVID-19 diagnosis. However, understanding the long-term kinetics of the nAbs levels and its correlation with disease severity is beyond the scope of this study. Because of limited specimen volumes, a few samples were not available for comparison across all commercial assays. Despite these limitations, our results were consistent with the manufacturer's performance claims and with the previous studies that had evaluated the accuracy of this cPass ELISA-based neutralization antibody assay.^8,16,22 

In conclusion, the cPass neutralization antibody assay demonstrated excellent performance characteristics and correlated well not only qualitatively but also quantitatively with PRNT titers, which are the gold standard. The IgG signal noise output on the commercially available serology platforms, Abbott, Euroimmun, and Siemens, had varying degrees of association with the nAb signals. However, positive IgG results by the commercial serology assays exhibited a close qualitative agreement with SARS-CoV-2 neutralization activity, making them a practical alternative in the absence of the cPass neutralization antibody assay.

More studies are required to adequately assess the protection against infection and longevity of the postvaccination response; nevertheless, nAbs are currently viewed as the first line of protective immunity.⁵  Although the IgG serology assays exhibited the ability to predict the neutralization activity against the SARS-CoV-2 virus in a reasonable manner, these are indirect approaches and lack the capacity to distinguish the binding from the nAbs. The cPass neutralization antibody assay offers the most direct and specific format for the detection of the neutralization function with capabilities for scalability, high throughput, and faster turnaround times. The accurate and specific detection of nAbs using easy-to-measure assays like the cPass assay may have a wide range of applications in population monitoring postinfection and postvaccination, and in the screening of donors for COVID-19 convalescent plasma therapy. Despite these advantages, emerging SARS-COV-2 variants raise concerns about resistance to nAbs in response to infections and vaccines.^35–37  Mutations of concern have been reported in the S protein with hotspots specifically in the RBD domain.^36,38  Further studies will be required to understand the ability of the assay to quantify the neutralization activity of antibodies targeted to the variant-specific RBD regions.