Global Study on the Accuracy of Human Epidermal Growth Factor Receptor 2-Low Diagnosis in Breast Cancer Open Access

This study is a nonclinical assessment of proficiency of real-world pathologists in diagnosing low levels of HER2 expression in breast cancer tissue. It was conducted in 2022 with the primary endpoint of distinction between HER2-low (IHC 1+ and 2+/ISH−) and HER2 IHC 0 in breast cancer samples with the secondary endpoint of measurement of the influence of a training program for correct identification of HER2-low cases.

Study cases were obtained from a HER2 expression study by IHC in breast cancer (AstraZeneca R&D, Cambridge, United Kingdom) with 500 commercial samples encompassing comprehensive range of IHC scores (0, 1+, 2+, 3+) stained with both Ventana PATHWAY 4B5 (monoclonal antibody) and Dako HcT (polyclonal antibody) at an independent central laboratory.¹⁹ 

The presented study aimed at pathologist’s proficiency of HER2 scoring according to ASCO/CAP guidelines (2018),⁴  which apply to any breast cancer sample irrespective of type and site. Accordingly, no samples obtained from metastases had been specifically included. However, almost one third of study cases represented small biopsies, which in terms of size compare to those usually taken from metastatic sites.

Digitized and anonymized images from this study (not related to other AstraZeneca or Daiichi-Sankyo clinical studies) focusing the case distribution on HER2-low expression, as per ASCO/CAP 2018 guidelines,⁴  were made available to Discovery Life Sciences Biomarker Services GmbH (DLS, Kassel, Germany). Two sample sets comprising 50 cases each were compiled for HcT- or 4B5-stained tumors to be scored by the participating pathologists through Pathotrainer virtual microscope software (Pathomation BV, Antwerp, Belgium, a CellCarta service). Another sample set (n = 25) was developed for training.

The study of HER2-low scoring proficiency comprised 3 steps: baseline pathologist scoring of cases (first assessment), virtual pathologist-to-pathologist training, and rescoring after a 2-week washout-period (second assessment) (Figure 1). The entire project was overseen by a steering committee of 8 independent experts in HER2 breast pathology (I.O.E., M.E.H.H., A.L., R.Y.O., F.P.-L., F.R., J.R., G.V.) advising on each study step defining the number of test and training samples, content of training, data interpretation, and publication. Complex/borderline cases were adjudicated by this expert panel to reach a consensus score.

Eighty pathologists were contacted from different laboratories, globally, by the sponsor (Diaceutics plc, Belfast, United Kingdom). They were informed of the study concept, design (Figure 1), and requirement to use a digital pathology platform to interpret HER2 images using ASCO/CAP 2018 scoring criteria.⁴ 

Study and training cases were selected according to the scoring scheme initially introduced by the FDA in 1998,²  then adopted and further developed by ASCO/CAP (version 2018)⁴  (Figure 2). Analogous to the HER2-low term, the IHC 0 category was named HER2-0.¹⁸  Within this IHC 0 group, pathologists were asked to also record any membrane staining in ≤10% invasive tumor cells (>no staining<1+). Intensity scoring was based on application of the magnification rule as introduced by the DLS group.^20,21 

For most study cases (61 of 65) HER2 amplification status was determined by using silver Ventana HER2 Dual ISH (DISH) DNA Probe Cocktail assay (conformité européenne–marked in vitro diagnostic) on a BenchMark ULTRA IHC/ISH system according to manufacturer’s protocol. Tumors with HER2/chromosome enumeration probe 17 (CEP17) ratio ≥2.0 and an average HER2 copy number of ≥4.0 signals/cell were considered amplified.

Study and training cases initially interpreted by 1 AstraZeneca pathologist were reevaluated and where appropriate, rescored by 2 DLS pathologists with extensive experience in HER2 interpretation (J.R., H.-U.S.).

In the absence of a validated reference standard distinguishing HER2-0 from HER2-low, the steering committee adjudicated cases considered challenging (14 of 50 cases) to determine a consensus score.

Taking DISH data into consideration, only those cases that could clearly be assigned to 1 of the 3 different HER2-classes (ie, HER2-0, HER2-low, and HER2-positive) were included in the final data evaluation.

On the Pathotrainer pathology platform, hematoxylin-eosin–stained, isotype negative control, and HER2-stained slides were provided for each case along with the 2018 ASCO/CAP guideline.⁴  A screen suitability check was implemented to guarantee a standardized view of digitized images. Cases were classified by both IHC scoring category (IHC 0, >no staining<1+, 1+, 2+, 3+) and percentage of tumor cell positivity at each intensity level.

According to the study design (Figure 1), participants were provided a proficiency assessment round of 50 stained cases (either 4B5 or HcT) that were completed during the course of 2 weeks. A week after finalization of the first proficiency assessment round, all participants were trained in groups of 10–20 within 4 weeks by 2 trainers (5 by J.R., 1 by C.A.). Each live virtual training, including slides from both 4B5 and HcT, was done on 1 day for 4 hours using an established format (see Supplemental Table 1 in the supplemental digital content, containing 4 tables and 2 figures at https://meridian.allenpress.com/aplm in the May 2025 table of contents.).²² 

Finally, cases that had been either underscored (classifying HER2-low as HER2-0) or overscored (classifying HER2-0 as HER2-low) by more than 10% of pathologists were reviewed for specific tissue- or staining-related issues that could underlie respective misscoring.

The extent to which raters assigned the same score to the consensus HER2 category (proficiency level) was measured by Cohen κ and rater agreement statistics where Cohen κ measures interrater reliability (>0.8: almost perfect/excellent; 0.61–0.8: substantial/good; 0.41–0.6: moderate; <0.41: fair, slight, and poor). Consensus intervals were determined at the 95% level accordingly.^23–25  Degree of interobserver variability was assessed by overall rater agreement (ORA) and related metrics (ie, positive percentage agreement [PPA], measuring how often the presence of the corresponding HER2 category was correctly identified, and negative percentage agreement [NPA], assessing how often the absence of the corresponding HER2 category was correctly identified).²⁶  To demonstrate the effect of training, receiver operating characteristic (ROC) curve analysis was performed using R (R Core Team 2022).²⁷  Results are considered excellent for area under the curve (AUC) values of 0.9–1, good for 0.8–0.9, and fair for 0.7–0.8. Lower values indicate a poor (0.6–0.7) or failed test (0.5–0.6). Significance level was set to P < .05 and calculated by χ² test. Samples considered not evaluable by a participant were excluded from calculations.

A total of 77 pathologists agreed to participate; 49 evaluated Ventana 4B5 and 28 Dako HcT–stained cases for the first assessment. Two pathologists could not participate in the second assessment and the data plausibility check disclosed another reader with a systematic recording error, resulting in 74 pathologists with evaluable results (Figure 1). Participating pathologists represented 14 countries with highest representation from Europe (n = 41) and North America (n = 23) followed by Asia/Pacific (n = 8) and South America (n = 2) (Supplemental Table 2).

The 2 study sets eligible for proficiency analysis (n = 46 by 4B5 and n = 45 by HcT) were comparable with respect to specimen type (resections versus biopsies, 72% versus 28%), histological tumor type (83% versus 87% not otherwise specified, 17% versus 13% lobular), and tumor grade (1 versus 2 versus 3, 15% versus 70% versus 15%, respectively). Distribution between the 3 categories of HER2-positive, HER2-low, and HER2-0 was in the expected range for 4B5 assay: 13% versus 52% versus 35%.¹¹  Since DISH data was not available at the time of case enrollment only IHC data was used for case selection. Therefore, the corresponding score distribution for HcT differs by doubling of HER2-positives and decrease of HER2-low cases: 27% versus 33% versus 40%.

In the HER2-low category, baseline pretraining agreement rates between expert consensus and 77 pathologists from 14 countries were >80%. After training, NPA improved from 80.6% to 91.1%. In the HER2-0 category, PPA (74.6%) was the only baseline parameter below the 80% benchmark but also improved to 89.2% by training (P < .001; Table 1). To assess the effect of training more specifically on the interpretation of HER2-low expression, ROC curve analysis and assessment of agreement rates were done. These disclosed a positive effect of training, especially for participants using 4B5. The AUC showed excellent performance values for HER2-positive diagnostics with both tests (≥0.9), and was highest with 4B5. Identification of HER2-0 and HER2-low was good (AUC 0.8–0.9) with both tests and was improved by specific instruction, especially in 4B5 users (Figure 3, A through F). Furthermore, training showed a positive effect within all 3 new HER2 categories for both assays according to the Cohen weighted κ (Supplemental Tables 3 and 4).

With respect to the historical FDA classification scheme comparing IHC 3+/2+ and IHC 0/1+ cases, overall posttraining rater agreement of 92.2% and 89.8% was achieved for both 4B5 and HcT, respectively (Supplemental Figures 1 and 2). Consensus was highest for ASCO/CAP classification into HER2-negative and HER2-positive (IHC3+ and IHC2+/amplified), both by κ statistics (almost perfect, >0.80) and ORA (92%–99%) (Supplemental Figure 2). Introduction of the new scoring category, HER2-low, caused a decrease of overall concordance to ORA of ∼82%, whereas κ values still indicated a substantial interrater agreement (κ >0.74) (Supplemental Figures 1 and 2). No significant differences in HER2 scoring were observed regarding the 14 countries and regions (data not shown).

The 4B5-stained collective turned out to be most informative as it represented the expected range of about 50%–60% HER2-low cases. Underscoring (classifying HER2-low as HER2-0) was more frequently observed as compared to overscoring (classifying HER2-0 as HER2-low) with 12 versus 9 cases (first assessment), respectively. The latter could significantly be reduced to 5 cases by the current training method. Review of respective cases disclosed staining and tissue issues that were almost specific for either underscored or overscored tumor samples (Table 2). Most instances of underscoring (6 of 13 cases) occurred in weakly stained cases showing single-cell spread, such as of lobular type rendering overestimation of cell number (as only half of tumor area is covered tumor cells). In addition, weakly stained single tumor cells can easily be missed, as high magnification is needed for them to be detected (Figure 4, A). Another 3 cases showed patchy membrane staining which was erroneously excluded from scoring (Figure 4, B). All instances of overscoring occurred in cases bordering the 10% cutoff, such as by counting in nonspecific basal membrane–like or cytoplasmic staining (5 of 9 cases) (Figure 4, C). Another 2 cases showed distinct cell borders that could be mistaken as faint membrane stains (Figure 4, D). In 1 overscored and 1 underscored tumor, pronounced shrinkage artefacts were observed; in such cases steering committee members recommended to consider the slide not evaluable and to restain or select a different tumor block if available.

This study is the first of its kind to systematically analyze the accuracy of HER2 diagnosis in breast cancer tissue by a large number of pathologists (n = 74) from 14 countries covering academic institutions (n = 35), hospital pathology labs (n = 27), and private practices (n = 12). Agreement rates, posttraining, for the HER2-low category between expert consensus and the real-world pathologists were 82.9% (4B5) and 84.8% (HcT), which are between the acceptable ORA percentage of ≥80% for test evaluation and an ideal ORA percentage of ≥90%.^28–29  The agreement rates of HER2-low are lower than the currently recommended 90% for HER2-positive¹⁷  but higher than those when HER2-positive was introduced decades ago and reported in the observations of the German Breast Group studies (GeparTrio to GeparSepto), where discordance rate between local and central testing decreased from 52% to 8.4% during a period of 12 years (2005–2017).³⁰  Although there is currently no global consensus available regarding the optimal agreement rate for HER2-low, it is the steering committee’s consensus that the 80% agreement should be practically acceptable at this early stage of clinical actionability for 1+ and 2+/ISH−. For example, Clinical Laboratory Improvement Amendments of 1988 passing requirements in proficiency testing are set at 80% in general³¹  and CAP also uses this criterion, for instance, by selecting HER2 test cases only if reference laboratories achieve at least 80% consensus for individual tissue cores.²⁹  As noted earlier, NPA was improved from 80.6% to 91.1% by a 4-hour training program. PPA (74.6%) in IHC-0 category was the only parameter below the 80% benchmark at baseline but was also improved to almost 90% (89.2%) after 4 hours of training. Thus, training significantly improved pathologists’ proficiency in identifying breast carcinomas that do not belong to HER2-low category (true negatives) and are instead HER2-0.

We also analyzed the agreement rates in current ASCO/CAP and historical FDA classifications (see Figure 2). The pathologists showed high level of accuracy in the diagnosis of the ASCO/CAP categories of HER2-negative versus HER2-positive of up to 98% by 4B5 and 92% by HcT, irrespective of training. Similarly, Cohen κ concordance values were above 0.8.

Within the ASCO/CAP and 3-tier HER2-classifications, overall agreement rates show marginal nonsignificant numerical increases after training. This may reflect the fact that the accuracy of HER2-positive (2+/ISH+ and 3+) is at an almost perfect level and may have masked the improvement of HER2-low and HER2-0. The missing training effect on HcT is most likely related to the 2-fold–high prevalence of HER2-positives in this sample set.

These agreement rates are overall higher than those described previously,^16,32  where participants were not informed about the intention of the study, did not receive specific training modalities, and were told no details about the assays used. Another recent study³³  on 105 HER2-negative breast cancer biopsies (stained with 4B5) demonstrated that consensus between 16 pathologists was higher if only HER2-null cases (without any staining) were grouped against cases with any staining (>no staining<1+, IHC 1+, and IHC 2+/nonamplified). The most relevant factor underlying interobserver disagreement in that study was poor reproducibility for cases bordering the 10% cutoff. Therefore, authors recommended using the “all or nothing” principle originally mentioned in the ASCO/CAP 2007 guideline.²⁸  In our series we also found that borderline status was the most significant cause of reader disagreement, accounting for ∼60% of all discordances. Regrouping alone did not improve agreement rates (data not shown), though we did identify specific staining issues that contributed to overscoring or underscoring (Figure 4). Some of these, such as heterogeneity and cytoplasmic staining, have been described as important reasons of discordance among 16 expert pathologists from the United Kingdom and Ireland as well.³⁴ 

In contrast to other recent studies,^33–36  this is the first to provide globally representative data about the level of ambiguity in HER2-low testing in the real world. It also shows that systematic training, making use of the magnification rule^20,21  for reproducible intensity scoring and including demonstration of specific challenging staining patterns, is a valuable tool to improve HER2-low scoring. We do expect an even higher effect if future training is focused on addressing the main sources of discordance (Table 2).

Future training should also take into consideration published HER2 scoring guidelines, of which the first have recently been edited in France³⁷  followed by Germany³⁸  and the United Kingdom.³⁹  Besides scoring issues, the control of preanalytical factors such as quality of fixation and the staining procedure itself is critical for accurate patient identification, and the implementation of on-slide controls that reflect the various staining intensities can further aid in staining quality control.^40–41 

Due to the approval of T-DXd in HER2-low breast cancer, international guidelines have recently been updated (June 2023)^17,18  to reflect this new treatment category. The ASCO-CAP guideline update does not endorse creating a new HER2-low result category for pathology reporting but acknowledges the need for pathologists to include a comment describing patient eligibility for therapy in HER2-low cases and the adoption of best laboratory practices for patient identification. Some of these best practices include the use of high magnification (×40) to interpret slides, secondary pathology review for borderline cases, and controls with a range of protein expression to distinguish between the 0 and 1+ cutoff.¹⁷  A European Society for Medical Oncology consensus statement addressing HER2-low breast cancer provided a recommendation for pathology participation in training and education programs to better report HER2-low.¹⁸  It also emphasized the need for pathologists to use a properly validated assay for the detection of HER2-low.¹⁸  Both guidelines also highlighted the increased importance of preanalytical variables in this new category.

To achieve more reproducible HER2-low scoring, one may consider more refined technologies such as digital image analysis,^42,43  for which CAP guidelines are already available.⁴⁴  However, before recommending new assays and technologies for HER2-low diagnostics,¹⁶  one should note that current clinical trial data about efficacy of antibody-drug conjugates in HER2-low breast cancer have been obtained by using a validated and widely used assay (4B5) with well-established ASCO/CAP scoring guidelines.^4,6 

The limitations of this study include the following: (1) digital images were used for scoring, which is not a standard clinical practice; (2) no clinical trial cases were used; and (3) consensus scoring was determined by expert pathologists, but not the pathologists of the central testing laboratory used to read for the DESTINY-Breast04 trial.

In conclusion, the global level of diagnostic accuracy for diagnosing breast cancer with low levels of HER2 expression is within an acceptable range but is still not optimal. Short-term training could significantly improve the accuracy of HER2 testing, and sources for misscorings could be delineated being specific for either underscoring or overscoring.

Special thanks to Judy Yu, PhD, a former AstraZeneca employee, who provided expertise and technical insights to support the study. Amy Hanlon Newell, PhD, Andrew Livingston, BS, and Victoria de Giorgio-Miller, MD, provided insightful comments. Sharon Allen oversaw the project management. Ouzna Morsli, MD, provided guidance for the study. Under the guidance of authors, assistance in editorial support was provided by ApotheCom and was funded by Daiichi Sankyo.