Learning From Failure: Negative Trials in Oncology

Dear Editors,

Our group of young oncologists understands the importance of speaking up about the message conveyed in Dr. Kurzrock's editorial, “Of Mice, Not Men: When the Bench-to-Bedside Bridge Is Broken,” and in her recent JIPO interview (https://t.co/Dq2uDgJHOc).^1–3  This thought-provoking article sparked intense discussions within our community, as it highlights a subject that is frequently on everyone's mind but rarely addressed.

As aspiring scientists in the field of oncology, we initially believed that success in our careers would be defined by our ability to cure, help, or alleviate the suffering of our patients. This belief led us to understand that conducting clinical trials would ultimately translate to better treatment decisions for patients, thereby leading into the success we value.

Unfortunately, we are observing a decrease in enthusiasm for academic medicine, with scientists' knowledge being reduced to a p-value, and their efforts deemed a failure if their study produces negative results. As young oncologists in research, we believe this can only improve if we change our mindset about how we measure success in science. Negative results are a crucial part of the educational learning process and selection of the most effective treatment. Expecting every study to yield positive outcomes would be impractical. Learning comes with mistakes and, in this case, negative surprises.

It is important for scientists to be informed of unsuccessful trials. Transparency and a clear publication strategy can help ensure effective future trial design. The tendency to publish positive results may create a cumulative management bias among scientists, which could negatively impact trial design leading to ineffective outcomes. Thus, negative trials can drive scientific advancement by identifying areas of further study.

Negative studies refer to those in which the intended primary endpoint was not achieved, however, there is no standardized definition of a negative trial. An analysis of 107 cancer-related studies⁴  found that positive trial outcomes were defined as experimental therapy showing a favorable outcome whereas negative trails are defined by results in favor of standard therapy with statistical significance or if the trial fails to meet an endpoint. The study found that both negative and positive trials had the same rate of publication; however, the positive trials were more likely to be published in high-impact journals and had higher mean citation rates over a 20-year period.

It was previously demonstrated that over 81 % of the phase 3 studies had lower success rates compared to the preceding phase 2 studies.⁵  Gyawali et al.⁶  recently reported that 18 indications for 10 cancer drugs failed to meet the primary endpoint in the post-approval trials resulting in label withdrawals in 61% of these indications. With the increase in accelerated approvals, especially in cancers with limited treatment options, a higher number of negative post-approval trials could be expected. The publication bias against the negative trials also seems to be amplified against the studies from low-middle-income countries (LMICs). A recent study by Wells et al. ⁷  found that clinical trials from LMICs have lower pharma incentives, while these trials have a higher chance of being practice changing. With over half of phase 3 clinical trials on cancer resulting as negative, the two top-tier cancer journals did not publish any negative studies.^{8, 9} 

The Journal of Immunotherapy and Precision Oncology (JIPO) carried out a non-scientific twitter poll on negative trials (Figure 1). The results were consistent with Dr. Kuzrock's article. As of December 12^th, 2022, there were a total of 82 responses. Over 90% of responders agreed that scientific journals do not publish negative results as frequently as positive ones, and that scientists who publish negative results are not considered as successful as those who publish positive results. Similarly, 94.7% of JIPO's Twitter respondents believe that professional societies do not select abstracts with negative results as often as those with positive results. Furthermore, 76.5% of responders expressed concern that negative trials may be detrimental to academic promotion.

Perhaps most importantly, negative trials impact patients and their families.²  Thus, the patients have the right to be informed and publishing the trial results is an act of respect and responsibility of the investigating team.¹⁰ 

Our suggestion is to implement a general policy requiring the publication of every negative trial in accordance with FDA regulations, along with the positive pre-clinical study that led to the trial design. This idea is also influenced by previous reviews. It could help prevent future efforts in treatments that have already proven to be non-beneficial. Therefore, there is an urgent need to improve the visibility and venues for negative trials in the future. Regardless of the outcome of a trial, this policy would give us back the most important sense of success, that is the purpose of improving human lives.