Anticyclic citrullinated peptide antibodies are important serologic markers for the diagnosis of rheumatoid arthritis. Several kinds of test reagents for automated immunoassay systems have been developed and used in recent years.
To evaluate the analytic and diagnostic performance of the new ADVIA Centaur anticyclic citrullinated peptide assay (Siemens Healthineers, Erlangen, Germany) compared with the Elecsys assay (Roche Diagnostics, Mannheim, Germany).
A total of 576 serum samples were collected from subjects, including 156 patients (27%) with rheumatoid arthritis. Precision performance and analytical measurement range for the ADVIA assay were evaluated. Diagnostic performance of the 2 assays was compared based on sensitivity, specificity, and area under the receiver operating characteristic curves.
The ADVIA assay showed a within-laboratory imprecision of 3.4% coefficient of variation for levels of 3.36 and 24.99 U/mL. This assay was demonstrated to be linear from 0.4 to 180.0 U/mL. With default cutoff values, sensitivity and specificity for diagnosing rheumatoid arthritis were 71.2% and 97.9%, respectively, for the ADVIA assay and 73.1% and 96.9%, respectively, for the Elecsys assay. With the best cutoff values from the analyses of the receiver operating characteristic curve, the sensitivity of the 2 assays was the same at 75.6%. However, the specificity of the ADVIA assay was 96.4%, whereas that of the Elecsys assay was 94.3%. The area under the receiver operating characteristic curve value for the ADVIA assay was 0.867, which was not significantly different from that of the Elecsys assay (0.865).
The ADVIA Centaur anticyclic citrullinated peptide assay showed good analytic and diagnostic performance in diagnosing rheumatoid arthritis.
Rheumatoid arthritis (RA), one of the most common autoimmune diseases, is characterized by inflammation of the synovial joints with progressive damage, eventually causing severe disability and reduced life expectancy.1 Prevalence and incidence of RA vary among different populations, affecting about 0.5% to 1.0% of the total population worldwide.2
In the past, the goal of RA treatment was to reduce pain and decrease the inflammation of affected joints. However, based on research conducted during the past decades, early and aggressive use of disease-modifying antirheumatic drugs such as hydroxychloroquin and methotrexate can reduce joint damage and irreversible disability, thus improving clinical outcome.3,4 Therefore, early diagnosis of the disease is the key to successful RA treatment. However, RA is a heterogeneous clinical condition. There is no gold standard for RA diagnosis. Accordingly, RA has been defined by using classification criteria based on a combination of several indicators. Traditionally, the rheumatoid factor (RF) was the only serologic marker for RA diagnosis and was included in the 1987 American College of Rheumatology (ACR, Atlanta, Georgia) criteria for RA classification.5 The RF is detected in approximately 50% to 80% of patients with RA. However, the RF is also elevated in patients with other rheumatic disorders or infectious diseases and can even be elevated in healthy, older individuals.6 Meanwhile, other tests targeting antikeratin antibodies or antiperinuclear factor have been introduced. However, clinical use of these new tests is limited because of their low sensitivity and the inconvenience of performing indirect immunofluorescence assays.7
Although the 1987 ACR criteria for RA classification was useful for diagnosing patients with RA who had advanced disease, the criteria were less helpful in diagnosing early RA.8 Therefore, a new classification system for RA was needed. In 2010, the ACR and the European League Against Rheumatism (EULAR, Zurich, Switzerland) published a new classification system for RA diagnosis. In the 2010 ACR/EULAR criteria for RA diagnosis, anticitrullinated protein antibodies (ACPAs) were newly included.9 Schellekens et al7 discovered RA-specific anticitrullinated peptide antibodies (anti-CCP antibodies) based on an enzyme-linked immunosorbent assay (ELISA) in 1998. The anti-CCP antibodies are autoantibodies for antigens containing the unusual amino acid citrulline in the rheumatoid joint. The first generation of anti-CCP assays showed greater specificity than RF. Thus, the anti-CCP assay has been widely used for RA diagnosis. Since then, a second generation of anti-CCP antibodies tests, with improved clinical diagnostic performance and even greater specificity, was introduced in 2002. Finally, a third generation of anti-CCP assays has been developed,10 which is generally used nowadays.
The anti-CCP assays using the ELISA principle are good diagnostic tools for distinguishing RA from other diseases. However, the ELISA method has several limitations, including being difficult to automate, having a narrow detection range, and requiring a long assay time. Accordingly, several test reagents that use automated immunoassay systems such as assays based on electrochemiluminescence immunoassays have been developed and used in recent years.11
Recently, the ADVIA Centaur anti-CCP assay (Siemens Healthineers, Erlangen, Germany), which is based on a chemiluminescence immunoassay, has been developed and introduced to clinical laboratories. The objective of this study was to evaluate the laboratory and clinical performance of this new anti-CCP assay and to compare the results with those of the previously developed Elecsys Anti-CCP assay (Roche Diagnostics, Mannheim, Germany).
MATERIALS AND METHODS
Assays
The anti-CCP concentration of sera from all study subjects was measured with a Roche Cobas e 602 immunoassay analyzer module and the Elecsys Anti-CCP kit. This assay uses the electrochemiluminescence immunoassay principle for quantitative detection of anti-CCP immunoglobulin G (IgG) autoantibodies in human serum. The anti-CCP concentration is reported in units per milliliter, and its analytical measurement range, according to the manufacturer, is suggested to be 7.0 to 500.0 U/mL. Default cutoff for the diagnosis of RA is 17 U/mL.
The anti-CCP IgG levels from the same subjects were also quantified with an ADVIA Centaur XPT immunoassay system and the ADVIA Centaur anti-CCP assay kit. This assay uses the chemiluminescence immunoassay principle, and its results are also reported in units per milliliter. The linearity range of ADVIA Centaur anti-CCP assay, according to the manufacturer, is from 0.4 to 200.0 U/mL, with a cutoff value for RA diagnosis suggested by the manufacturer to be 5 U/mL.
C-reactive protein (CRP) levels were determined with the UniCel DxC 800 Synchron Clinical System with SYNCHRON CRP reagents (Beckman Coulter Inc., Brea, California). Erythrocyte sedimentation rate (ESR) was measured with the TEST 1 analyzer (Alifax, Polverara, Padova, Italy). The RF was quantified with a Dimension Vista System with RF Flex reagent cartridge (Siemens Healthineers).
Laboratory Performance Evaluation
Precision performance of the ADVIA Centaur anti-CCP assay was assessed based on guidelines from the Clinical and Laboratory Standards Institute (Wayne, Pennsylvania) document EP5-A2. Testing was performed using a single lot of reagents, calibrators, and controls with one analyzer. Two levels of quality control (QC) materials were assayed, in replicates of 2, at 2 separate times per day for 20 days (ie, 80 tests per QC level). Manufacturer-claimed (Siemens Healthineers), within-laboratory imprecision performance for mean anti-CCP levels of 5.24 and 34.44 U/mL were coefficients of variation (CVs) of 6.14% and 3.53%, respectively.
Tests for validating the analytical measurement range of the ADVIA Centaur anti-CCP assay were performed based on the Clinical and Laboratory Standards Institute document EP6-A. The anti-CCP Master curve material (Siemens Healthineers) with 5 preassigned levels of 0.4, 45, 90, 135, and 180 U/mL was tested in duplicate, and the linearity was evaluated by comparing against expected concentrations calculated from the linear fit and statistically best polynomial fit.
Study Subjects
Serum samples from 576 patients who visited National Health Insurance Service Ilsan Hospital (Goyang, Republic of Korea) for arthralgia were collected from October 2016 to January 2017. All specimens were sent to the Department of Laboratory Medicine for anti-CCP assays. These sera were separated immediately upon arrival and subjected to the Elecsys anti-CCP assay. Thereafter, residual samples were stored at −70°C until testing with the ADVIA Centaur anti-CCP assay. Medical records for all enrolled subjects, including age, sex, and CRP, ESR, and RF levels, were reviewed retrospectively. Among the 576 patients, 156 (27%) were diagnosed with RA by rheumatology specialists, based on the 2010 ACR/EULAR classification criteria for RA.9 This study was approved by the institutional review board of Ilsan Hospital (approval 2016-06-011).
Statistical Analyses
All statistical analyses were performed using Analyse-it method evaluation edition software (version 3.76.1, Analyse-it Software, Leeds, United Kingdom). Comparisons between study groups were performed using a Mann-Whitney U test for continuous variables or a χ2 test for categorical variables. Correlation coefficient between different assay results was determined by the Spearman rank test. The area under the receiver operating characteristic curve (AUROC) was used to discriminate patients with RA from those without RA. Best cutoff value for each anti-CCP assay was decided as the value showing the maximum Youden index by receiver operating characteristic curve analysis. Logistic regression analysis, with the presence of RA as the binary dependent variable and levels of RF and anti-CCP as predictor variables, was performed to calculate the predicted probability value of the 2-marker combination. This value was also used to estimate the AUROC value for the 2-marker combination. P values of less than .05 were considered statistically significant for all analyses.
RESULTS
Laboratory Performance of the ADVIA Centaur Anti-CCP Assay
Mean concentrations for 2 QC materials were 3.36 and 24.99 U/mL. The assay showed a within-laboratory imprecision of 3.4% coefficient of variation (CV) for each QC level. Repeatability, between-run precision, and between-day precision are summarized in Table 1.
Precision Performance of the ADVIA Centaur Anticyclic Citrullinated Peptidea Assay During a 20-Day Evaluation Period (n = 80 for Each Level)
The ADVIA Centaur anti-CCP assay range was demonstrated to be linear from 0.4 to 180.0 U/mL. The CVs from duplicates for each preassigned level were between 0.0% and 1.9%. Statistically, the best-fit regression was a third-order polynomial equation in the linearity test. When expected concentrations calculated from the linear fit were compared with their respective concentrations with a third-order polynomial fit, the percentage of difference for each assessed level ranged from −1.0% to 4.1%. The linearity equation was y = 1.017x + 1.158 (r2 = 0.9996).
Characteristics of Study Groups
Age, sex, and concentrations of assessed biomarkers for RA and non-RA groups are shown in Table 2. Except for the proportion of female patients, all other parameters, including levels of biomarkers, were higher in the RA group than they were in the non-RA group.
Correlation of Anti-CCP Levels With Other Laboratory Results
The RF levels were moderately correlated with anti-CCP concentrations (Elecsys assay, r = 0.654; ADVIA Centaur assay, r = 0.639) (Table 3). The anti-CCP levels determined by the Elecsys assay were highly correlated with those measured by the ADVIA Centaur assay (r = 0.857; 95% CI, 0.833–0.878) (Table 3; Figure). Regarding qualitative results for anti-CCP, the 2 assays were concordant in 97.0% of all cases. The κ coefficient between the 2 assays was 0.91 (95% CI, 0.87–0.95) (Figure).
Correlation Between Anticyclic Citrullinated (Anti-CCP) Peptide Assays and Other Laboratory Results
Correlation between the 2 anticyclic citrullinated peptide (anti-CCP) assays. The correlation coefficient for results between the 2 anti-CCP assays was 0.857 (n = 576; 95% CI, 0.833–0.878; P < .001) by Spearman rank test. The percentage of agreement of qualitative results by the 2 assays was 97.0% (95% CI, 95.3%–98.1%), and the κ coefficient was 0.91 (95% CI, 0.87–0.95). Numbers of patients with rheumatoid arthritis (RA) and patients without RA according to the qualitative results from the 2 assays are presented in each quadrant.
Correlation between the 2 anticyclic citrullinated peptide (anti-CCP) assays. The correlation coefficient for results between the 2 anti-CCP assays was 0.857 (n = 576; 95% CI, 0.833–0.878; P < .001) by Spearman rank test. The percentage of agreement of qualitative results by the 2 assays was 97.0% (95% CI, 95.3%–98.1%), and the κ coefficient was 0.91 (95% CI, 0.87–0.95). Numbers of patients with rheumatoid arthritis (RA) and patients without RA according to the qualitative results from the 2 assays are presented in each quadrant.
In addition, there was a weakly positive correlation between anti-CCP levels and ESR (Elecsys assay, r = 0.295; ADVIA Centaur assay, r = 0.300). The CRP levels were not significantly correlated with anti-CCP concentrations (Elecsys assay, r = 0.080; ADVIA Centaur assay, r = 0.108).
Diagnostic Performance of the Anti-CCP Assay
Table 4 shows the AUROC values for the laboratory tests including the 2 anti-CCP assays in discriminating the patients with RA from the non-RA group. The AUROC of the ADVIA Centaur assay was 0.867, which was significantly (P < .001) greater than the CRP values (0.597) or the ESR values (0.738). However, the AUROC of the ADVIA Centaur was not statistically (P = .10) different from that of the RF value (0.836). The Elecsys assay showed a similar (P = .85) AUROC value of 0.865 compared with that of the ADVIA Centaur assay. When anti-CCP level as determined by the ADVIA Centaur assay was combined with the RF level, the AUROC was 0.890, which was significantly (P = .04) greater than that of the anti-CCP level determined by the ADVIA Centaur assay alone.
Area Under Receiver Operating Characteristic Curve Values of Laboratory Tests for Rheumatoid Arthritis Diagnosis
Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the 2 anti-CCP assays are summarized in Table 5. When calculating diagnostic performance using the default cutoff values suggested by manufacturers, the clinical sensitivity for an RA diagnosis was 0.731 with the Elecsys assay and 0.712 with the ADVIA Centaur assay, and the clinical specificities were 0.969 and 0.979, respectively. When using respective best cutoff values from AUROCs, these 2 assays showed the same sensitivity (0.756). However, the specificity of the ADVIA Centaur assay was 0.964, whereas that of the Elecsys assay was 0.943.
DISCUSSION
Several autoantibodies such as RF, anti-Sa, and ACPA including antikeratin antibodies and antiperinuclear factor can be detected in patients with RA. Among them, RF and ACPA are commonly used in RA diagnosis because of their diagnostic and prognostic values.12 Moreover, RF and ACPA are present for months to years before disease onset. Because ACPAs are known to be more specific for RA, they have been widely used in clinical practices since the early 2000s. They were included in the 2010 ACR/EULAR for RA diagnosis. The present study evaluated laboratory and clinical performance of the new ADVIA Centaur anti-CCP assay. In addition, patient age, sex, and levels of ESR, CRP, and RF were reviewed retrospectively.
In this study, within-laboratory imprecision of the ADVIA Centaur anti-CCP assay for mean levels of 3.36 and 24.99 U/mL were both 3.4% CV. This assay is thought to have acceptable precision performance because the within-laboratory imprecision claimed by the manufacturer are 6.14% and 3.53% CV at mean levels of 5.24 and 34.44 U/mL, respectively. The percentages of difference at 5 preassigned levels ranging from 0.4 to 180.0 U/mL were between −1.0% and 4.1%. Therefore, the assay was demonstrated to be linear in that range.
ESR and CRP are acute-phase reactants. They have been used in the classification criteria for RA. Therefore, we analyzed the AUROCs of ESR and CRP for RA diagnosis as well as correlation coefficients between results of those biomarkers and anti-CCP assays. Values for ESR and CRP were increased in patients with RA compared with the values from patients without RA (Table 2). However, there were low or no correlations between ESR or CRP levels and anti-CCP concentrations, although RF levels showed a moderate correlation with anti-CCP concentrations in our study subjects. Meanwhile, AUROC values of the 2 anti-CCP assays tested in our study were similar to that of RF (Table 4). In addition, anti-CCP level determined by the ADVIA Centaur assay improved the diagnostic performance slightly when it was combined with RF in our study. In a previous study, 87 among 295 patients (29.5%) clinically defined with RA were negative for RF whereas 30 of these 87 patients (34.5%) showed positive results for the anti-CCP assay.13 Thus, concurrent use of anti-CCP and RF assays might be helpful in the diagnosis of patients with RA.
High sensitivity and specificity of an assay are always important in accurate diagnosis of a single affected individual, particularly when clinical manifestations or results of other diagnostic markers are confusing. In this study, we analyzed the sensitivity, specificity, PPV, and NPV at not only the default cutoff but also at the best cutoff for each anti-CCP assay. The best cutoff for each anti-CCP assay, as determined by maximum Youden index, was 9.0 U/mL for the Elecsys anti-CCP assay and 1.9 U/mL for the ADVIA Centaur anti-CCP assay. Both best cutoff values were lower than the cutoffs suggested by their respective manufacturers (Table 5). Sensitivity, specificity, PPV, and NPV of the Elecsys assay at the default cutoff were 0.731, 0.969, 0.898, and 0.906, respectively. At the best cutoff value, sensitivity and NPV were slightly increased (sensitivity, 0.756; NPV, 0.912), whereas specificity and PPV were slightly decreased (specificity, 0.943; PPV, 0.831). The ADVIA Centaur anti-CCP assay also showed similar changes in sensitivity and specificity at the best cutoff value, compared with those at the default cutoff. Previous studies have also evaluated the same Elecsys anti-CCP kit and reported that sensitivities and specificities at the default cutoff of 17 U/mL ranged from 0.677 to 0.906 and from 0.868 to 0.984, respectively.11,14–16 Best cutoff values in those studies were also less than or equal to the manufacturer's cutoff. At those cutoffs, sensitivities and specificities were 0.688 to 0.922 and 0.862 to 0.968, respectively. Similar to our results, sensitivities for the Elecsys assay were slightly increased when using the best cutoffs, whereas specificities were decreased. Therefore, clinical laboratories might need to evaluate diagnostic performance of the anti-CCP assay they use, so that appropriate cutoff value can be applied for their own purposes.
To our knowledge, no study has yet reported the diagnostic performance of the ADVIA Centaur anti-CCP assay. Thus, results from our study on the diagnostic performance of the ADVIA Centaur anti-CCP assay will help clinical laboratories determine which anti-CCP assay to use for RA diagnoses. In our study, the ADVIA assay showed a sensitivity of 0.712 and a specificity of 0.979 in discriminating patients with RA from subjects with arthralgia at the default cutoff. The AUROC of the same assay was 0.867. In a previous meta-review, which summarized 12 studies on the diagnostic accuracy of anti-CCP assays, the area under the curve for the discrimination of 1988 patients with RA from 1740 disease controls without RA was 0.817 based on pooled sensitivities and specificities of 0.634 and 0.955, respectively.17
Our study has a limitation in that we could not estimate diagnostic performance of the evaluated assays according to RA stages because we were unable to obtain sufficient clinical information on patients with RA, including the degree of arthralgia and the number of affected joints because of the retrospective study design. We could not investigate the correlation between results of the anti-CCP assays and clinical manifestations either. In further studies, the diagnostic performance of different anti-CCP assays needs to be evaluated and compared with each other according to patients' symptoms and disease stage.
In conclusion, the new ADVIA Centaur anti-CCP assay showed good analytic and diagnostic performance and demonstrated comparable results to the Elecsys anti-CCP assay. The ADVIA assay can be helpful in making RA diagnoses, and clinical laboratories need to establish their own cutoff values for effective RA diagnosis with an anti-CCP assay.