How to Validate Predictive Immunohistochemistry Testing in Pathology?

To the Editor.—I read with interest a recent editorial by Erik Thunnissen entitled “How to Validate Predictive Immunohistochemistry Testing in Pathology? A Practical Approach Exploiting the Heterogeneity of Programmed Death Ligand-1 Present in Non-Small Cell Lung Cancer.”¹  It would be significant and desirable if laboratories could simplify different spheres of validation of predictive immunohistochemistry (IHC) biomarkers. The introduction of programmed death ligand-1 (PD-L1) testing for immunotherapy started a new, more complex era for IHC assay development and validation. The challenges that laboratories face forced us to rethink what type of laboratory test IHC is, what “fit-for-purpose” validation means, and a few other parameters that were first embraced by drug development and pharmaceutical research rather than by anatomic pathologists.²  In following their steps, we have discovered how to distinguish analytic/technical sensitivity and specificity from diagnostic sensitivity and specificity, and that this distinction is essential in the validation of predictive biomarkers.^3,4  Thunnissen explored the role of what he termed “critical samples, which have an epitope concentration close to the threshold of the validated assay,” the type of samples that were traditionally used by proficiency testing programs to assess calibration/analytic sensitivity of the IHC assays, and were also previously termed “descriptive limit of detection” and incorporated in IHC Critical Assay Performance Controls (iCAPCs) in 2015.³  These samples are derived from human tissues or other sources (cell lines, xenografts, etc) and can be used to demonstrate basic analytic sensitivity, specificity, and reproducibility as described for iCAPCs.⁴  However, testing of 20 positive and 20 negative samples still applies for technical validation because their purpose is not to show analytic sensitivity, but that the assay protocol performs as it should in a representative set of clinical samples (eg, specific tumor type) by demonstrating reportable range, cellular localization, tissue distribution, results with the representative range of preanalytic conditions, etc.⁵  Furthermore, any clinical validation, including “indirect clinical validation” that is used by the author in this editorial, requires at least demonstration of assay diagnostic accuracy in comparison with an established reference standard (or “diagnostic accuracy criteria”) and, for the purpose of comparison of methods, at least 50 positive and 50 negative samples are recommended by the CLSI DP12-A2 User Protocol for Evaluation of Qualitative Test Performance.⁶  The purpose of the indirect clinical validation is to ensure that the candidate test has the same or nearly the same diagnostic accuracy as the comparator assay (reference test/diagnostic accuracy criteria), and therefore that it is safe for patient selection for a specific therapy. Although analytic sensitivity and specificity are related to diagnostic sensitivity and specificity, no studies have yet demonstrated that we can make direct assumptions from one to the other. Although calibration and optimization of the assay are greatly helped with “critical samples” (aka iCAPCs), to benchmark analytic sensitivity, unfortunately these samples do not tell us too much about diagnostic accuracy and are not sufficient for indirect clinical validation. I am concerned that taking the shortcut approach for indirect clinical validation for predictive biomarkers could compromise patient safety.