Introduction

 Use of the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) during chemotherapy is associated with decreased hospitalization rates, improved quality of life, and longer survival. Limited data exist on the benefit of this symptom assessment tool for monitoring immune-related adverse events (irAEs).

Methods

 We incorporated irAE-related items from the National Cancer Institute’s (NCI) PRO-CTCAE in a trial evaluating ipilimumab in combination with androgen deprivation therapy in 16 patients with hormone-sensitive prostate cancer. For comparison, NCI’s CTCAE version 4.0 was used by clinicians.

Results

 IrAE-related PRO-CTCAE surveys and matched CTCAEs (184 pairs) reporting abdominal pain, diarrhea, fatigue, anorexia, nausea, vomiting, rash, and pruritus were collected at each treatment administration and during follow-up. Fatigue, diarrhea, rash, and pruritus were the symptoms most frequently reported by both patients and clinicians. Agreement was lowest for pruritus (κ = 0.10) and highest for rash (κ = 0.64). IrAEs were more commonly reported and of higher grade with PRO-CTCAE scores compared with CTCAE grades.

Conclusion

 PRO-CTCAEs focused on irAEs capture the patient’s immunotherapy experience while complementing the clinician’s toxicity assessment measures. Further study is needed to assess PRO-CTCAE’s utility in identifying and managing irAEs.

Immune checkpoint inhibitors (ICIs) can improve survival across a range of malignancies[1] and, in a meta-analysis, were associated with fewer high-grade (grade ≥ 3) adverse events (AEs) based on the Common Terminology Criteria for Adverse Events (CTCAE) relative to chemotherapy.[2,3] However, high-grade immune-related AEs (irAEs) can have a substantial negative impact on the patient. Given the potential severity of irAEs, which if left untreated can compromise quality of life and reduce a patient’s chance of benefiting from treatment,[4] it is essential that they are recognized early.

To better capture the patient experience, the National Institutes of Health and the United States Food and Drug Administration support incorporating patient-reported outcome (PRO) measures through direct patient reporting in oncology trials.[5] To do so, the PRO version of CTCAE (PRO-CTCAE) was developed by the National Cancer Institute (NCI), which includes a library of 124 items representing 78 symptomatic toxicities reflected in the CTCAE. Measurement of selected PRO-CTCAE items may be incorporated into clinical trials based on anticipated symptomatic toxicities.[6] Use of the PRO-CTCAE has been validated in a large trial of patients receiving chemotherapy and/or radiation therapy. Its use alongside routine care is associated with fewer hospitalizations and improved survival relative to those receiving routine care alone.[7,8]

Limited data are available about the benefit of using PRO-CTCAE to monitor irAEs resulting from treatment with ICIs in clinical trials.[9,10] To address this, we incorporated selected items from the NCI’s PRO-CTCAE into a phase II clinical trial (ClinicalTrials.gov Identifier: NCT02020070) evaluating the efficacy and safety of ipilimumab, a humanized monoclonal antibody directed to the cytotoxic T-lymphocyte–associated protein 4 (CTLA-4) in combination with standard androgen deprivation therapy (ADT) in patients with hormone-sensitive prostate cancer (HSPC). HSPC may have immunologic differences compared with castration-resistant prostate cancer, providing a therapeutic opportunity for ICI in this earlier disease setting. The PRO-CTCAE items chosen represented the symptoms and related severities most often associated with ICI treatment. A customized survey of 14 questions covering the previous 7 days was incorporated into each clinic visit. This study aimed to evaluate whether the addition of PRO-CTCAE to standard care during oncology clinical trials can lead to a more comprehensive understanding of patients’ experience of irAEs.

All patients provided written informed consent to participate in this single-center study, which was approved by the institutional review board and conducted in accordance with the Good Clinical Practice guidelines and the Declaration of Helsinki. This study analyzed survey data from patients being treated during a phase 2 clinical trial for HSPC at a tertiary care center between August 2014 and January 2017. All patients enrolled in the clinical trial were eligible and included in this study. Cohort A included patients with de novo metastatic HSPC. Cohort B included those with biochemically recurrent or metastatic HSPC with a prior history of radical prostatectomy, a rising serum prostate-specific antigen (PSA) level 1 ng/mL or higher, and a PSA doubling time of 12 months or less. Patients in both cohorts received up to four doses of ipilimumab and 8 months of ADT. Patients in cohort A also underwent radical prostatectomy. Patients with a history of autoimmune disease were excluded.

Ipilimumab 10 mg/kg was initially used but resulted in a higher-than-expected irAE rate in the first six patients. The study protocol for the trial was amended, with the dose reduced to 3 mg/kg, but the trial was terminated early for an unfavorable risk:benefit ratio. However, because the PRO-CTCAE analysis was prespecified in the approved protocol, we conducted the analysis. The PRO-CTCAE survey consisted of 14 questions from the NCI’s PRO-CTCAE library, covering abdominal pain, diarrhea, fatigue, anorexia, nausea, vomiting, rash, and pruritus, with correspondent attributes of frequency, severity, and interference in daily activities (as described in the Supplemental file, available online). The recall period for PRO-CTCAE was 7 days. The results were registered via paper surveys and collected at each treatment administration and during follow-up visits up to 84 weeks. Treating clinicians used NCI’s CTCAE version 4.0 to monitor symptoms. The results of PRO-CTCAE were not used for clinical decision-making, such as dose adjustments or discontinuation. For comparison purposes, reports of AEs were matched between PRO-CTCAE and corresponding CTCAE terms, as previously defined.[6,11]

For statistical analyses, all-grade and high-grade rates of matched AEs captured by PRO-CTCAE and CTCAE assessments were summarized and reported descriptively. Grade 3 or higher on CTCAE was considered “high grade,” which corresponded to a 3 or 4 score on a 0 to 4 scale on PRO-CTCAE. For items measuring the interference associated with a particular symptom, a 3 corresponded to “quite a bit” and 4 “very much,” while for those items measuring severity, a 3 corresponded to “severe” and 4 “very severe.” For items reflecting the frequency of a symptom, a 3 corresponded to “frequently” and 4 “almost constantly.” This mapping has previously been reported.[12,13] Patient adherence was calculated as the percentage of PRO-CTCAE surveys completed at required clinic visits. Agreement between PRO-CTCAE and CTCAE was quantified by Cohen’s kappa coefficients using the “Kappa” function of the “vcd” package in R Studio version 1.2.5001 (RStudio, Inc., Boston, MA). Kappa coefficients were calculated based on the presence/absence of each symptomatic AE derived from PRO-CTCAE and CTCAE assessments, pooling within patients across all time points.

Sixteen patients were treated with ipilimumab and ADT as follows: seven in cohort A and nine in cohort B. Median age at enrollment was 63 years (range, 46–74 years), and median serum PSA level was 2.03 ng/mL (range, 0.15–26.83 ng/mL). All patients had an ECOG of 0 to 1. Twelve (75%) had a Gleason score of 9 to 10. There were 184 matched pairs of PRO-CTCAE and CTCAE evaluations—75 from cohort A and 109 from cohort B. Median duration of treatment was 22.0 weeks (95% CI, 8.8–23.9) for cohort A and 27.3 weeks (95% CI, 16.5–30.3) for cohort B. The median number of PRO-CTCAE surveys completed during active treatment was seven (range, 2–9) for cohort A and nine (range, 3–9) for cohort B. Adherence was 88.0% in cohort A and 99.1% in cohort B.

Fatigue, diarrhea, rash, and pruritus were the most commonly reported (Figs. 1A, B). Table 1 summarizes the symptom ratings at the patient level for both the PRO-CTCAE scores and CTCAE grades across all time points. A comparison of the patient and physician reporting of diarrhea showed it to have occurred “frequently” or “almost constantly” in 43.8% of patients (n = 7), whereas clinicians reported grade 2 (“increase of 4–6 stools per day over baseline”) and grade 3 (“increase of ≥ 7 stools per day over baseline”) occurred in 25% (n = 4) and 12.5% (n = 2) of patients, respectively (Table 1). Overall, 93.8% (n = 15) of patients reported diarrhea using PRO-CTCAE compared with 62.5% (n = 10) reported by clinicians using CTCAE. Four (25%) patients received systemic steroids to treat immune-related diarrhea, two for grade less than 3 by the CTCAE, after other supportive measures failed. The percentage of patients reporting diarrhea “frequently” or “almost constantly” by PRO-CTCAE was 75% in those treated with steroids (n = 3) and 66.7% in those who did not receive steroids (n = 4).

Figure 1

Frequency of reporting of fatigue, diarrhea, pruritus, and rash by clinicians (left, by CTCAE) and by patients (right, by PRO-CTCAE). (A) Selected symptoms captured by clinicians and reported by patients in cohort A. CTCAE, Common Terminology Criteria for Adverse Events; PRO-CTCAE, Patient-Reported Outcomes version of the in Common Terminology Criteria for Adverse Events. (B) Selected symptoms captured by clinicians and reported by patients in cohort B. CTCAE, Common Terminology Criteria for Adverse Events; PRO-CTCAE, Patient-Reported Outcomes version of the in Common Terminology Criteria for Adverse Events.

Figure 1

Frequency of reporting of fatigue, diarrhea, pruritus, and rash by clinicians (left, by CTCAE) and by patients (right, by PRO-CTCAE). (A) Selected symptoms captured by clinicians and reported by patients in cohort A. CTCAE, Common Terminology Criteria for Adverse Events; PRO-CTCAE, Patient-Reported Outcomes version of the in Common Terminology Criteria for Adverse Events. (B) Selected symptoms captured by clinicians and reported by patients in cohort B. CTCAE, Common Terminology Criteria for Adverse Events; PRO-CTCAE, Patient-Reported Outcomes version of the in Common Terminology Criteria for Adverse Events.

Close modal
Table 1

CTCAE and PRO-CTCAE: Assessment of symptoms and agreement between results

CTCAE and PRO-CTCAE: Assessment of symptoms and agreement between results
CTCAE and PRO-CTCAE: Assessment of symptoms and agreement between results

Agreement between PRO-CTCAE scores and CTCAE grades, assessed by Cohen’s kappa coefficients, ranged from 0.10 (no agreement) to 0.64 (good agreement).[14] Pruritus had the lowest agreement (κ = 0.10; 95% CI, −0.09 to 0.29; p = 0.31), and rash had the highest (κ = 0.64; 95% CI, −0.01 to 1.28; p = 0.05) (Table 1). There were also statistically significant, moderate levels of agreement between CTCAE nausea and PRO-CTCAE nausea frequency and severity (both κ = 0.43; 95% CI, 0.09–0.77; p = 0.01). Kappas could not be calculated between CTCAE fatigue and PRO-CTCAE fatigue; however, agreement was high between patients and clinicians when reporting the presence/absence of fatigue (100% for PRO-CTCAE fatigue severity, 87.5% for PRO-CTCAE fatigue interference).

The PRO-CTCAE is designed to complement the clinician-reported CTCAE and to inform key stakeholders (e.g., clinicians, patients, regulatory authorities) of the safety and tolerability of therapy from the patient’s perspective.[3,15] Our results demonstrate that clinicians generally assessed ipilimumab-related irAEs as lower grade and of lower frequency than patients’ direct reporting of their experiences.[16] This aligns with other studies evaluating both targeted agents and chemotherapy. Veitch and colleagues[17] assessed the full PRO-CTCAE library of symptoms at three time points for 243 patients enrolled in phase 1 trials and found that overall patient–clinician agreement for individual symptomatic AEs ranged from poor to moderate. Discordance was driven by underreporting of symptoms by clinicians; however, it is also critical to note that there are inherent differences between the CTCAE rating scale (which provides a grade) and PRO-CTCAE (which provides a score). Of note, the CTCAE grade often bundles symptom severity, frequency, and interference, in contrast to PRO-CTCAE scoring, which separates these components by design. This also highlights the importance of viewing these measures as complementary and recognizing limitations when trying to make direct comparisons between these instruments. While the instruments may correspond to the same symptom (e.g., fatigue), what clinicians are being asked to grade using CTCAE is not identical to what patients are being asked to report using the PRO-CTCAE. Hence, these instruments were not designed to be directly compared.

In this study, pruritus had the lowest agreement between CTCAE and PRO-CTCAE and rash the highest. This is concordant with the published literature demonstrating a higher agreement for “observable symptoms” and a lower agreement for more subjective symptoms.[16] A systematic review of dermatologic toxicities associated with ICIs highlighted clinicians’ inadequate assessment of impact on quality of life from rash and/or pruritis.[18] A study in patients with melanoma receiving ICI was not able to demonstrate a decline in severe AEs in patients using PRO-CTCAE versus standard-of-care reporting; however, it was incumbent on patients to call their physician’s office with symptoms.[19] Alternatively, real-time electronic feedback from PRO-CTCAE reporting was associated with lower hospitalization rates in patients receiving chemotherapy.[7,8] This automated reporting to clinicians in real-time of patient-reported toxicity may account for differences between studies and may indicate the optimal implementation of this tool to impact patient care and enhance communication. This study did not use PRO-CTCAE for clinical decision-making, such as treatment discontinuation or dose reduction.

The use of PRO measures is highly relevant when considering drug safety not only in early-stage trials but also for agents in the late stages of development to help inform our understanding of drug tolerability in the context of the overall benefit of a particular therapy. Interestingly, CTCAE and PRO-CTCAE may provide different insights into the patient experience. In a longitudinal study of patients with advanced lung cancer, Basch and colleagues[20] found that clinician reporting of CTCAE symptoms and low Karnofsky Performance Status was statistically significantly associated with unfavorable clinical outcomes, such as emergency room visits or death. In contrast, patient reporting of the same symptoms was not. However, patients’ reports of symptoms had higher concordance with validated measures of daily health status than clinicians’ reports. When taken together, these perspectives provide a more complete picture of the patient experience. It is striking, though perhaps not surprising, that when viewing the distribution of reporting for select items on the CTCAE and PRO-CTCAE (Figs. 1A, B) in this study, the patient-reported measures appear to apply a broader range of severity in contrast to clinician reporting. Although the toxicities in phase 1 trials may be difficult to predict—making the implementation of PRO-CTCAE in early-stage trials more challenging—the toxicities of anti-CTLA4 antibodies were well established before this trial, allowing for the feasibility of a brief PRO-CTCAE survey focused on irAEs. The format provided in the clinic may have increased compliance in our study, but there are clear advantages to electronic reporting with real-time feedback when this tool is being evaluated more frequently outside of clinic visits.

This study was in the context of a small trial in prostate cancer, which limits generalizability and interpretation; efficacy and full clinician-reported safety data were not part of this manuscript. Additionally, pooling within-patient data across all assessment times may have spuriously elevated the kappas. The kappa CIs were often wide because of the small sample size, and there was no adjustment for multiple comparisons. Recall bias represents another limitation. We selected PRO-CTCAE questions that reflected some of the more common irAEs, but it was not inclusive of all irAEs (e.g., neurologic), some of which are not amenable to PRO-CTCAE reporting (e.g., asymptomatic transaminitis).

Although a review of our customized PRO-CTCAE survey results by clinicians was not mandatory in the clinical trial, we believe that PRO-CTCAE could be leveraged to increase clinician awareness of patients’ experience and enhance patient–provider communication. Further studies using PRO-CTCAE with direct feedback to clinicians are needed to fully evaluate the potential for this symptom tool in irAE management.

The authors thank Margaret McPartland (Memorial Sloan Kettering Cancer Center) for editorial support.

Supplemental materials are available online with the article.

1.
Emens
LA,
Ascierto
PA,
Darcy
PK,
et al
Cancer immunotherapy: opportunities and challenges in the rapidly evolving clinical landscape
.
Eur J Cancer
.
2017
;
81
:
116
129
.
2.
Magee
DE,
Hird
AE,
Klaassen
Z,
et al
Adverse event profile for immunotherapy agents compared with chemotherapy in solid organ tumors: a systematic review and meta-analysis of randomized clinical trials
.
Ann Oncol
.
2020
;
31
:
50
60
.
3.
Arnaud-Coffin
P,
Maillet
D,
Gan
HK,
et al
A systematic review of adverse events in randomized trials assessing immune checkpoint inhibitors
.
Int J Cancer
.
2019
;
145
:
639
648
.
4.
Brahmer
JR,
Lacchetti
C,
Schneider
BJ,
et al
Management of immune-related adverse events in patients treated with immune checkpoint inhibitor therapy: American Society of Clinical Oncology Clinical Practice Guideline
.
J Clin Oncol
.
2018
;
36
:
1714
1768
.
5.
Kluetz
PG,
Slagle
A,
Papadopoulos
EJ,
et al
Focusing on core patient-reported outcomes in cancer clinical trials: symptomatic adverse events, physical function, and disease-related symptoms
.
Clin Cancer Res
.
2016
;
22
:
1553
1558
.
6.
Correspondence between PRO-CTCAE and CTCAE terminologies
.
Division of Cancer Control & Population Sciences;
National Cancer Institute; U.S. Government. Last updated Apr 26, 2023. Accessed Mar 2, 2023
. healthcaredelivery.cancer.gov/pro-ctcae/terms.html
7.
Basch
E,
Deal
AM,
Kris
MG,
et al
Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial
.
J Clin Oncol
.
2016
;
34
:
557
565
.
8.
Basch
E,
Deal
AM,
Dueck
AC,
et al
Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment
.
JAMA
.
2017
;
318
:
197
198
.
9.
Tolstrup
LK,
Bastholt
L,
Zwisler
AD,
et al
Selection of patient reported outcomes questions reflecting symptoms for patients with metastatic melanoma receiving immunotherapy
.
J Patient Rep Outcomes
.
2019
;
3
:
19
.
10.
Steffen McLouth
LE,
Lycan
TW,
Jr.,
Levine
BJ,
et al
Patient-reported outcomes from patients receiving immunotherapy or chemoimmunotherapy for metastatic non-small-cell lung cancer in clinical practice
.
Clin Lung Cancer
.
2020
;
21
:
255
263.e4
.
11.
Basch
E,
Reeve
BB,
Mitchell
SA,
et al
Development of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE)
.
J Natl Cancer Inst
.
2014
;
106
.
12.
Watson
GA,
Veitch
ZW,
Shepshelovich
D,
et al
Evaluation of the patient experience of symptomatic adverse events on Phase I clinical trials using PRO-CTCAE
.
Br J Cancer
.
2022
;
127
:
1629
1635
.
13.
Filho
OM,
Giobbie-Hurder
A,
Lin
NU,
et al
A dynamic portrait of adverse events for breast cancer patients: results from a phase II clinical trial of eribulin in advanced HER2-negative breast cancer
.
Breast Cancer Res Treat
.
2021
;
185
:
135
144
.
14.
McHugh
ML.
Interrater reliability: the kappa statistic
.
Biochem Med (Zagreb)
.
2012
;
22
:
276
282
.
15.
Kim
J,
Singh
H,
Ayalew
K,
et al
Use of PRO measures to inform tolerability in oncology trials: implications for clinical review, IND safety reporting, and clinical site inspections
.
Clin Cancer Res
.
2018
;
24
:
1780
1784
.
16.
Basch
E,
Iasonos
A,
McDonough
T,
et al
Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study
.
Lancet Oncol
.
2006
;
7
:
903
909
.
17.
Veitch
ZW,
Shepshelovich
D,
Gallagher
C,
et al
Underreporting of symptomatic adverse events in phase I clinical trials
.
J Natl Cancer Inst
.
2021
;
113
:
980
988
.
18.
Walocko
FM,
Ly
BC,
White
MS,
Chen
SC,
Yeung
H.
Health-related quality of life measures and immune checkpoint inhibitors: a systematic review
.
J Am Acad Dermatol
.
2020
;
82
:
1004
1006
.
19.
Tolstrup
LK,
Bastholt
L,
Dieperink
KB,
et al
The use of patient-reported outcomes to detect adverse events in metastatic melanoma patients receiving immunotherapy: a randomized controlled pilot trial
.
J Patient Rep Outcomes
.
2020
;
4
:
88
.
20.
Basch
E,
Jia
X,
Heller
G,
et al
Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes
.
J Natl Cancer Inst
.
2009
;
101
:
1624
1632
.

Source of Support: This work was supported in part by funds from the National Institutes of Health/National Cancer Institute to Memorial Sloan Kettering Cancer Center (P30CA008748) and David H. Koch Fund for Prostate Cancer Research. Ipilimumab was supplied by Bristol-Myers Squibb. The funding agencies had no role in the design of the study, the collection, analysis, or interpretation of the data; the writing of the manuscript; or the decision to submit the manuscript for publication.

Conflict of Interest: Daniel Vargas P. de Almeida reports speaker compensation from Astellas, AstraZeneca, Bayer, Eli Lilly, Janssen-Cilag, and Sanofi-Aventis. Daniel C. Danila reports research support from US Department of Defense, American Society of Clinical Oncology, Prostate Cancer Foundation, Stand Up to Cancer, Janssen Research & Development, Agensys, Genentech, CreaTV; consultant for Angle LLT, Axiom LLT, Janssen Research & Development, AstraZeneca, BioView LTD, Clovis, Astellas, Medivation, Pfizer, and Agensys. Michael J. Morris reports travel support from Endocyte, Fujifilm; research funding (to institution) from Novartis, Bayer, Celgene, Corcept, Roche/Genentech, Janssen; is an uncompensated consultant to Bayer, Novartis, Johnson and Johnson; is a compensated consultant to Oric, Lantheus, Curium, AstraZeneca, Amgen, Daiichi, Convergent Therapeutics, Pfizer, and ITM Isotope Technologies. Wassim Abida reports speaking honoraria from Roche, Medscape, Aptitude Health, Clinical Education Alliance, MJH Life Sciences and touchIME, consulting fees from Clovis Oncology, Janssen, ORIC Pharmaceuticals, Daiichi Sankyo and AstraZeneca UK, and research funding (to institution) from AstraZeneca, Zenith Epigenetics, Clovis Oncology, Glaxo-Smith Kline, and ORIC Pharmaceuticals. Howard I. Scher reports compensated consultant/advisor to Ambry Genetics Corp, Konica Minolta Inc, Bayer, Pfizer Inc, Sun Pharmaceuticals, WCG Oncology; uncompensated consultant/advisory to Amgen, Janssen Research & Development, LLC, Janssen Biotech, Inc,; honoraria from Arsenal Capital; has received research funding (to institution) from AIQ Solutions, Epic Sciences, Illumina, Inc, Janssen, Menarini Silicon Biosystems, Prostate Cancer Foundation and ThermoFisher; supported in part by NIH/NCI grant P50 CA092629 (MSK SPORE in Prostate Cancer); intellectual property rights BioNTech, Elucida Oncology, MaBVAX, Y-mAbs Therapeutics, Inc.; non-financial support from Amgen, Bayer, Epic Sciences, Promontory Therapeutics (formerly Phosplatin), Pfizer Inc, Prostate Cancer Foundation, and WCG Oncology. Karen A. Autio reports research funding (to institution) from Pfizer, Amgen, Trishula, CytomX, Astra-Zeneca, Eli Lilly, Glaxo-Smith Kline, and Parker Institute for Cancer Immunotherapy. The remaining authors have no disclosures.

Supplementary data