Context.—

Standardized structured reporting (SSR) among pathologists results in more complete diagnoses and, subsequently, improved treatment decisions and patient outcomes. Therefore, SSR templates' usage is advocated in oncology guidelines. However, actual SSR usage varies widely. Previous studies have shown multiple impeding and facilitating factors regarding SSR implementation.

Objective.—

To select, develop, and test an evidence-based multifaceted strategy, tailored to the impeding and facilitating factors to improve SSR implementation in oncologic pathology.

Design.—

Six strategy elements to increase the use of SSR were selected on the basis of a barrier and facilitator analysis, literature review, and consecutive discussions with a nationwide expert panel and project team. In collaboration with a professional organization for developing SSR templates (PALGA), we developed elements and combined them in 1 multifaceted strategy and subsequently tested effectiveness and feasibility.

Results.—

The 6 strategy elements were as follows: (1) renewed Web site including SSR information; (2) e-learning including SSR instructions; (3) communication manual describing communication about SSR; (4) improved feedback process, including use of the “Feedback Button” within SSR templates and “Frequently Asked Questions” on the Web site; (5) information sheet on SSR updates within SSR templates; and (6) monthly telephone conversations to discuss audit and feedback information regarding local SSR usage. A significant change (12.4%) in SSR usage among test laboratories was noticed. After the first test, e-learning and the “Feedback Button” were deemed most feasible and effective. However, awareness of all elements could be increased.

Conclusions.—

Next steps will be to optimize the tailored strategy, to distribute it to all Dutch pathology laboratories, and to evaluate effectiveness and feasibility in a nationwide setting.

Improved completeness and accuracy of diagnostic oncologic reports are of substantial importance, since these reports form the basis of multidisciplinary oncology care.1,2  Standardized structured reporting (SSR), instead of narrative reporting, improves the quality of reporting in various medical disciplines, including pathology, radiology, endoscopy, and surgery316  and, as a consequence, the quality of multidisciplinary team meetings.16  SSR enables the reporting of specific data elements within predefined checklists based on evidence-based datasets.17  Moreover, secondary users such as health policy makers and cancer registries profit from the use of SSR by improved operability, shareability, and reusability of data elements.18 

In pathology, more completeness in relevant prognostic factors in reports has resulted in improved risk assessments and more appropriate treatment decisions.16,19,20  In particular, for patients with stage II colorectal cancer, we have shown a significant survival benefit.19  Therefore, the use of SSR in pathology is recommended in several international and national oncologic guidelines, and standard datasets are developed to enable SSR in pathology.2123 

Despite the obvious benefits of SSR and guideline recommendations, SSR is currently underused. Frequency of SSR usage varies widely, depending on tumor type and infrastructure.2429 

Previous studies based on the frameworks of Flottorp et al30  and Grol and Wensing31  have shown an extensive range of impeding and facilitating factors of SSR implementation among pathologists and other multidisciplinary team members as the receivers of SSR.32,33  According to Grol and Grimshaw,34  a comprehensive multifaceted implementation program tailored to the barriers and facilitators of SSR should be developed to improve its implementation.35,36 

Several strategy elements have been developed to improve implementation of SSR, including an intranet site, teleconferences, regional face-to-face meetings, symposia, discussions, workshops, training sessions, and audit and feedback.25,27  However, none of these strategy elements were developed from an extensive analysis of the barriers and facilitators of SSR in pathology, nor were these strategy elements tested before nationwide implementation. Therefore, the aim of this study was to improve the implementation of SSR in oncologic pathology in 3 phases: (1) by systematically selecting implementation strategy elements, based on a previous analysis of barriers and facilitators; (2) by developing a multifaceted strategy including these elements; and (3) by testing effectiveness and feasibility of this multifaceted strategy among pathologists to improve it before nationwide dissemination.

Our study consisted of the following 3 steps: (1) selection of digital implementation strategy elements, (2) development, and (3) effectiveness and feasibility tests. In this section, the methodology used in each step is described.

Setting

In the Netherlands, SSR templates have been developed by the nationwide network and registry of histopathology and cytopathology, also known as the PALGA foundation (PALGA; Pathologisch-Anatomisch Geautomatiseerd Landelijk Archief) since 2009 (Table 1). In this article, we refer to the PALGA foundation as “the professional organization for developing standardized structured reporting templates.” Currently, there are 30 templates available.37  The running software application (PALGA Protocol Modules) is linked within the laboratory information system. Our study focused on SSR templates used for urologic, gynecologic, and gastroenterologic oncologic diagnoses to limit the scope of the study. These templates cover most oncologic diagnoses, and major variation has been shown in the use of these templates among laboratories. The use of templates in 2019 varied between 41% and 70% for urologic tumor types, between 23% and 65% for gynecologic tumor types, and between 31% and 84% for gastroenterologic tumor types. The use of the SSR templates per laboratory are shown in Figure 1, A through C. Templates for colorectal biopsies and cervical cytology are mandatory for the national screening program and are therefore excluded. Detailed information about the included templates is provided in Supplemental Table 1 (see supplemental digital content containing supplemental data and 4 tables at https://meridian.allenpress.com/aplm in the December 2022 table of contents).

Table 1

Additional Information About the Professional Organization for Developing Standardized Structured Reporting Templates (PALGA)

Additional Information About the Professional Organization for Developing Standardized Structured Reporting Templates (PALGA)
Additional Information About the Professional Organization for Developing Standardized Structured Reporting Templates (PALGA)
Figure 1

A, National standardized structured reporting usage for the 2019 urology standardized structured reporting templates. B, National standardized structured reporting usage for the 2019 gastroenterology standardized structured reporting templates. C, National standardized structured reporting usage for the 2019 gynecology standardized structured reporting templates. A through C, Each symbol indicates a separate laboratory. Medians are indicated by the horizontal line. Abbreviation: SSR, standardized structured reporting.

Figure 1

A, National standardized structured reporting usage for the 2019 urology standardized structured reporting templates. B, National standardized structured reporting usage for the 2019 gastroenterology standardized structured reporting templates. C, National standardized structured reporting usage for the 2019 gynecology standardized structured reporting templates. A through C, Each symbol indicates a separate laboratory. Medians are indicated by the horizontal line. Abbreviation: SSR, standardized structured reporting.

Close modal

Step 1: Selection of Implementation Strategy Elements

Study Design and Population

The selection of the implementation strategy elements included in the toolbox was based on an extensive analysis of influencing factors and determinants among multiple stakeholders of SSR implementation conducted in previous studies.32,33  To further select implementation strategy elements to improve SSR implementation, we performed a literature search, organized a consensus meeting with a nationwide expert panel, and organized consensus meetings with the project team. The nationwide expert panel included representatives of the most important stakeholders regarding SSR implementation, particularly pathologists, oncology-treating clinicians, patient representatives, and database experts. Of each stakeholder type, 1 expert was already involved as an advisor during the preparation of the study application. The snowball method was applied to complement this group of advisors with additional experts. Ultimately, the nationwide expert panel consisted of 3 pathologists, 3 oncology-treating clinicians, 3 patient representatives (all related to the 3 domains of this study: oncologic urology, gynecology, and gastroenterology), 2 PALGA representatives, 2 Netherlands Comprehensive Cancer Organization representatives, 1 PALGA liaison, and 1 implementation expert. The project team consisted of a pathologist, PALGA representative, implementation expert, and researcher.

Data Collection and Analysis

First, after analysis of the barriers and facilitators, the literature was consulted on useful theoretical constructs, associated implementation strategy elements, and evidence for these strategy elements. The latter specifically focused on the literature about implementation strategy elements regarding (1) evidence-based guideline implementation in general, (2) information technology solutions in health care, and (3) SSR implementation. The search was limited to articles in English.

Second, a consensus meeting with the nationwide expert panel was organized to brainstorm about potential implementation strategy elements related to the most important influencing factors regarding SSR implementation.38  The meeting was chaired by a junior researcher with previous experience in chairing focus group discussions. Informed consent was given before the start of the meeting. At the start, the results of previous studies on barriers and facilitators of SSR and the literature search on potential implementation strategy elements were presented by the junior researcher. Next, in teams of 4, the experts discussed which influencing factors matched with which potential implementation strategy elements, both known from the literature and from the experiences of the experts. The experts then discussed the potential implementation strategy elements. Eventually, the junior researcher summarized the recommended implementation strategy elements, and additional remarks could be provided by the experts. The consensus meeting was audiotaped and transcribed verbatim. In addition, notes were taken by a senior researcher.

Third, multiple discussions with the multidisciplinary project team took place to discuss the final potential implementation strategy elements selected from the literature study and from the consensus meeting in order to achieve a toolbox that could be tested among 6 laboratories.

Step 2: Development of Implementation Strategy Elements

Study Design and Population

The different strategy elements were developed in close collaboration with the professional organization for developing SSR templates (PALGA) and with members of the multidisciplinary project team, and were directed to clinical pathologists. Before effectiveness and feasibility testing, the strategy elements were alpha tested for operability and functionality. In addition, for alpha testing of the strategy elements, we collaborated with a pathology resident and communications expert.

Data Collection and Analysis

After the selection phase, specific content of the implementation strategy elements was developed, based on multiple resources from the professional organization for developing SSR templates, such as the Web site showing information about SSR templates, the PALGA liaison instructions, and knowledge about SSR usage from involved SSR experts. We used a Web-based platform for development of the e-learning.39  At the end of the development phase, the strategy elements were alpha tested before the real-world testing. First, the content of the implementation strategy elements and their potential use in daily practice among a broad audience were assessed by presenting them during a yearly meeting with PALGA liaisons and a meeting of the SSR-template working group attended by pathologists. Second, 5 experts, 2 PALGA representatives, a pathologist, a pathology resident, and an implementation expert tested the functionality of the different implementation strategy elements. Third, these elements were tested for their usability and understandable content by 3 pathology experts and an additional expert in communications. Eventually, all digital strategy elements were combined in a single toolbox: a digital infographic including links to or additional explanatory information about the individual strategy elements.

Step 3: Effectiveness and Feasibility Test for the Implementation Strategy Elements

Study Design and Population

We tested the effectiveness and feasibility of the toolbox to determine potential pitfalls before scaling up to a national level by conducting a small-scale study in 6 laboratories.40,41  Laboratories were selected by region (north, middle, south), size (small, medium, large), type (internal general, internal university, external), and extent of use of SSR templates in 2018 (low, moderate, high). Two researchers of the project team visited the laboratories to provide additional information and to discuss study participation preconditions (having a PALGA liaison and being motivated to participate in this study). In addition, a contact person was designated and the restructured role of the PALGA liaison and the new role of the local opinion leader on SSR, called “SSR pathologist,” were allocated and explained. The medical manager of each laboratory signed a consent form to formalize their commitment and another form to authorize data collection from the PALGA database on SSR usage at the laboratory level. To assess the feasibility, we performed both an interview study with 2 pathologists per intervention laboratory and a survey among pathologists of the 6 intervention laboratories. Only pathologists using the included SSR templates in our study were included in the interviews and survey. To understand the effect, we determined and tested the changes in SSR usage among the 6 intervention laboratories compared to 34 control laboratories. The before measurement included all qualifying cases (all malignancies) between June 2019 and September 2019. The after measurement included all qualifying cases (all malignancies) between June 2020 and September 2020. Selection criteria for qualifying cases for effect evaluation are shown in Supplemental Table 2.

Data Collection

At the start, the toolbox was shared by email with pathologists of the participating laboratories by the study contact person. Pathologists were instructed to use the toolbox according to their own need. In 3 laboratories, pathologists were exposed to the toolbox for 8 months (February 2020 to September 2020), and at the other 3 laboratories, pathologists were exposed for 4 months (June 2020 to September 2020). Monthly telephone conversations were organized to inventory circumstances that may affect the test and to answer questions related to the test.

For assessing the effectiveness we used the indicator “Use of SSR.” Each pathology report that was created by using SSR was automatically flagged in the PALGA database. In this way, we were able to collect data on SSR usage. This was defined as the proportion of pathology reports composed by means of an SSR template.

Feasibility was tested in 2 phases, using qualitative and quantitative methods, including semistructured interviews and a survey at the end of the test period.4042  We conducted semistructured interviews with 2 pathologists per laboratory, who were recruited by the contact person of the laboratory, about their toolbox experiences. To preclassify these experiences, we developed an interview guide based on the first 4 categories of the feasibility framework of Bowen et al.42  Informed consent was given before the interviews. Interview participants reported characteristics, for example, age, sex, work experience, and type of SSR template use. The interviews lasted between 17 and 60 minutes and were audiotaped.

After the interviews, the toolbox was adopted and emailed to all participants by the study contact person. At the end of the study, all participants received an eSurvey based on the results of the interviews on feasibility from the study contact person to determine the feasibility of the implementation strategy elements among all participants. The eSurvey consisted of multiple questions regarding the actual use, experiences, and self-reported effectiveness of the implementation strategy elements. Furthermore, questions about the experienced barriers of SSR implementation and the most promising strategy elements for future improvement of SSR implementation were included. Two people from the project team, 1 pathology resident, 2 experts on feasibility studies, and 1 communication expert tested the eSurvey. The survey did not accept unanswered questions. The survey was available from October 6, 2020, until November 6, 2020, and weekly reminders were sent.

Data Analysis

We used IBM SPSS Statistics V25.0 (IBM Corp, Armonk, New York) to explore and test the effect of the intervention. First, laboratory characteristics (region, size, type, SSR use in 2018) were reported and the pathology report characteristic (type of SSR template) was extracted from the PALGA database. Second, descriptive statistics were used to gain insight into the proportion of the SSR that was used, as well as the laboratory and pathology report characteristics before and after distribution of the tailored strategy. Third, change in SSR use among the test laboratories and control laboratories was tested, taking into account the clustering of the cases within laboratories: We performed a multilevel logistic regression analysis by building a model, including covariates for strategy (none/tailored), measurement time (before/after), and its interaction term (strategy*measurement). The interaction term tests the difference in change between the test laboratories and control laboratories.

The interviews were transcribed verbatim for content analysis. Two researchers independently extracted experiences of the pathologists with the toolbox, using ATLAS.ti (version 8.3, ATLAS.ti Scientific Software Development, Berlin, Germany). These experiences were allocated to the first 4 categories of the feasibility framework of Bowen et al42 : demand, acceptability, practicality, and implementation barriers of the implementation strategy elements. Each encoder established an independent coding tree and discussed both coding trees to establish 1 refined coding tree. In case of discrepancies, codes were discussed to reach consensus.

We also used IBM SPSS Statistics V25.0 for the statistical analysis of the survey results. Missing data were coded as such and excluded from the data analysis. We performed descriptive statistics to determine the proportions of agreement of SSR users with the feasibility of the toolbox.

Ethical approval was obtained by the Human Research Committee (2018-4124).

Selection of Implementation Strategy Elements

By combining the barriers and facilitators of 2 previous studies, 18 different barriers and 14 facilitators were identified.32,33  For all these factors, an implementation strategy element was determined. Six strategy elements were considered short-term, that is, defined as being able to be developed before the start of the test phase. The remainders were long-term, owing to long-term technical or organizational implications, and fall outside the scope of the current article. In Table 2, an overview of the previously determined barriers and facilitators that are connected to the implementation strategy elements selected by the expert panel and project group is given. The developed and tested short-term implementation strategy elements are distinguished from the long-term elements.

Table 2

Barriers and Facilitators of Standardized Structured Reporting Addressed by Pathologists and Multidisciplinary Team Members Connected to Implementation Strategy Elements

Barriers and Facilitators of Standardized Structured Reporting Addressed by Pathologists and Multidisciplinary Team Members Connected to Implementation Strategy Elements
Barriers and Facilitators of Standardized Structured Reporting Addressed by Pathologists and Multidisciplinary Team Members Connected to Implementation Strategy Elements

Development of Implementation Strategy Elements

After selection, we developed a multifaceted strategy tailored to the barriers and facilitators found in previous research, which consisted of the following: (1) an improved Web site, (2) e-learning, (3) a communication manual, (4) an adapted feedback process, (5) a sheet within the SSR templates showing updates of SSR templates, and (6) monthly telephone conversations to discuss audit and feedback information regarding the use of different SSR templates within the laboratory. Key information about the implementation strategy elements is provided in Table 3. In Figure 2, the infographic of the toolbox is presented. Screenshots from all elements can be found in the Supplemental Data.

Table 3

Overview of Specifications of Implementation Strategy Elements According to Reporting Guidelinesa

Overview of Specifications of Implementation Strategy Elements According to Reporting Guidelinesa
Overview of Specifications of Implementation Strategy Elements According to Reporting Guidelinesa
Figure 2

Toolbox standardized structured reporting (SSR) implementation. Abbreviation: FAQ, frequently asked questions.

Figure 2

Toolbox standardized structured reporting (SSR) implementation. Abbreviation: FAQ, frequently asked questions.

Close modal

Renewed Web Site With SSR-Related Information

The aim of the adapted Web site containing information about the SSR templates was to inform SSR users on diverse aspects of the implementation of SSR templates. To develop a new version of the Web site showing information about SSR templates, we not only restructured the information already available, but also complemented it with information needed from multiple stakeholders regarding the SSR templates: the acreage of SSR templates, an explanation of the development and feedback processes of SSR templates, and the advantages of SSR usage. The latter was shown on the frontpage of the Web site section on SSR.

E-learning SSR Usage

The e-learning aimed to improve the knowledge and skills of SSR users regarding the SSR templates. A storyboard in PowerPoint was constructed and the design principles of Mayer and Moreno43  were applied to develop the e-learning modules. An overview of the different elements is provided in Supplemental Table 3. Each module focused on a learning goal based on the barriers and facilitators listed in Table 2. Next, we obtained advice from a senior researcher with experience developing e-learning for primary care. Eventually, the e-learning content was based on the documentation of the SSR templates, supplemented with instructional videos and challenging questions based on this documentation.37  Pathologists were free to select modules according to their own needs, considering the influence of prior experience using SSR.32 

Communication Manual (About Communication [Informing and Motivating] Between the Professional Organization Developing SSR Templates and Pathology Laboratories)

The communication manual aimed to support pathologists in their communication with their PALGA liaison and PALGA regarding newly available SSR templates, updates of SSR templates, settings of SSR templates, the feedback process of SSR templates, and the advantages of using SSR. In addition, pathologists were encouraged to discuss the content of the pathology reports with other multidisciplinary team members. The communication manual was based on a previous version of the description of the role and tasks of PALGA liaisons and was supplemented with a newly created role and task description of a local opinion leader of SSR, called “SSR pathologist.” The laboratories were free to select a local opinion leader, but the head of the pathology department was most often selected.

Feedback Procedure of SSR

The aim of the revised feedback process was to improve the transparency and efficiency of the current feedback process. To improve transparency, the feedback process of the SSR templates was published, for example, the experts participating in the PALGA SSR template working group, responsible for processing feedback. To improve efficiency, more emphasis was placed on the availability and additional value of the “Feedback Button,” instead of mailing or calling PALGA with feedback regarding the SSR templates. The feedback provider received an acknowledgment of receipt from PALGA, as well as information about the subsequent processing of the feedback. The Feedback Button streamlined the feedback mechanism and gave insights into perceived problems. This was supplemented with a “Frequently Asked Questions” page (part of the Web site) to deduct common questions and misconceptions of SSR users regarding SSR.

Additional Sheet Within SSR Templates on SSR Updates

The aim of the additional sheet within SSR templates was to improve the awareness of SSR users regarding new elements within the SSR template. The sheet was developed by the professional organization for developing SSR templates in the same matter as the other tabs within SSR templates. The additional sheet included information on various updates, both technical and content related, and the most recent template version is shown. During the test, more emphasis was placed on the availability and additional value of this tab.

Telephone Conversations About Audit and Feedback Information Regarding Local SSR Usage

During the test period, contact persons were called monthly to discuss audit and feedback information on local SSR usage, specifically the absolute numbers of SSR used per SSR template per laboratory retrieved from the PALGA database. Furthermore, these meetings could be used to recall the aim of the test study.

Evaluation of Effect

Study Population

In total, we included 31 052 pathology reports. The number of included pathology reports per period for both the test laboratories and the control laboratories is shown in Figure 3. Of the 31 052 included reports, 20 906 (67%) reported urologic oncology cases, 7306 (24%) reported gastroenterologic oncology cases, and 2840 (9%) reported gynecologic oncology cases. An overview of the characteristics of laboratories is provided in Supplemental Table 4.

Figure 3

Flow diagram of the tailored strategy distribution and qualifying pathology reports.

Figure 3

Flow diagram of the tailored strategy distribution and qualifying pathology reports.

Close modal

SSR Use

The change in use of SSR between after- and before-measurement period (Δ10.1) significantly differed between the test laboratories and the control laboratories (P < .001): The average use of SSR among all test laboratories increased by 12.4% (33.7–46.1, Δ12.4), whereas among the control laboratories, the average use of SSR increased by 2.3% (64.4–62.1, Δ2.3).

Evaluation of Feasibility

Study Population

For each laboratory, 2 pathologists were included in the interview study (n = 12), of which 4 were male and 8 were female, with an average age of 44 years (range, 35–60 years). The survey yielded 19 responses from 41 participants (46%): 11 female (58%) and 8 male (42%). The average age of the respondents was 44 years (range, 31–61 years), and their average work experience was 12 years (range, 1–31 years). Of the 19 respondents, 10 were specialized in gastroenterologic pathology, 6 were specialized in gynecologic pathology, and 4 were specialized in uropathology. All were familiar with SSR and used SSR templates in practice. Of the 19 respondents, 9 (47%) worked at an academic pathology laboratory, 8 (42%) worked in a nonacademic pathology laboratory, and 2 (11%) worked in an independent pathology laboratory.

Interviews

We interviewed 12 pathologists to test the feasibility of the toolbox, including 6 tailored implementation strategy elements. The experiences of the pathologists with each strategy element are illustrated with some quotes (Figure 4). The quotes are translated literatim from Dutch to English.

Figure 4

Quotes of the experiences of the participants. Abbreviations: PALGA, professional organization for developing standardized structured reporting templates; SSR, standardized structured reporting.

Figure 4

Quotes of the experiences of the participants. Abbreviations: PALGA, professional organization for developing standardized structured reporting templates; SSR, standardized structured reporting.

Close modal

Survey

Of the 19 respondents, 13 (68%) received the toolbox properly, and all of them (n = 12, 1 missing) looked at the content. The recommendation of toolbox use to colleagues varied, with a median of 7.0 (range, 2–9). According to 13 of the 19 respondents (68%), the provision of the toolbox could be improved by publishing the toolbox on the PALGA Web site, and, according to 10 respondents (53%), in the digital newsletter of the Dutch Pathology Association. In addition, 3 pathologists added the suggestion of a direct link to the toolbox included in the SSR templates.

Results on the actual use, self-reported effectiveness, and recommendation scores are provided in Table 4. The actual use of the 5 implementation strategy elements varied: the elements integrated in the SSR templates were used more often than elements only accessible via links in the toolbox. Respondents mentioned a lack of knowledge on the availability of specific implementation strategy elements, except for the sheet on SSR updates, as the main reason for not using them. The self-reported effectiveness of the respondents regarding the use of the implementation strategy elements to use SSR more often and in a better way varied between the elements: E-learning was considered most effective for using SSR more often, and both e-learning and use of the “Feedback Button,” for using SSR in a better way. The results on self-reported effectiveness were in line with the average recommendation score.

Table 4

Feasibility of Implementation Strategy Elementsa

Feasibility of Implementation Strategy Elementsa
Feasibility of Implementation Strategy Elementsa

Furthermore, 2 of 6 SSR pathologists (33%) were encouraged by the rates on current SSR use at the laboratory level, and 3 of 17 respondents (18%, 2 missing) were encouraged to use SSR by the PALGA liaison of their laboratory.

Remaining Barriers and Promising Strategy Elements of SSR Implementation

As some strategy elements were not yet developed, we also evaluated the remaining barriers and promising strategy elements of SSR implementation. Of the respondents, 13 of 19 (68%) still perceived barriers related to SSR. Most experienced barriers related to the content of SSR templates, specifically the mandatory items and their rigidity. As a result, more than half of the 17 respondents (2 missing) mentioned content-related improvements as most promising. Of these 9 respondents, 8 (89%) mentioned the revision of the content of SSR templates by expert panels as a promising strategy element. Furthermore, 8 of 17 respondents (47%) mentioned improvements in readability of the SSR template, of which 6 (75%) wanted feedback from oncology-treating clinicians. Additionally, 6 of 17 respondents (35%) wanted the possibility to receive audit and feedback information regarding performance indicators based on SSR output, such as tumor grading, and 5 of 17 (29%) wanted technologic improvements, of which 5 wanted automatic transfer of clinical information (100%) and 4, integration of speech recognition (80%).

To improve nationwide implementation of SSR in oncologic pathology, we selected implementation strategy elements, developed a tailored strategy including these elements, and tested its effectiveness and feasibility in a real-world setting to optimize the toolbox before nationwide expansion. We showed both effectiveness and feasibility in our pilot experiment.

Our multifaceted strategy consisted of 6 implementation strategy elements: an updated Web site, e-learning, a communication manual, a feedback procedure, a sheet with information on updates incorporated into the SSR templates, and a monthly telephone conversation to discuss audit and feedback information regarding SSR usage. The tailored strategy showed a significant increasing trend in SSR usage among test laboratories compared to control laboratories. The actual usage of specific implementation strategy elements was highest when they were already part of the SSR template itself but could be improved for all strategy elements by emphasizing availability through national dissemination strategies familiar to pathologists. The e-learning and revised feedback procedure, mainly the use of the “Feedback Button,” were most feasible, effective, and had the highest recommendation score in improving SSR usage. The pathologists' experiences regarding the feasibility and effectiveness of the renewed Web site, communication manual, “Frequently Asked Questions,” and sheet on the updates of SSR were inconclusive, suggesting variability in (information) requirements among pathologists. Before nationwide expansion, feedback on element-specific content has to be processed and methods to increase toolbox exposure should be questioned to conform to the pathologists' needs.

This study is the first to develop and test a tailored strategy primarily focused on increasing the implementation of SSR among pathologists. Previous research has focused on the implementation of SSR, mostly deployed nationwide strategies, to inform pathologists about SSR. However, our study showed differences between laboratories and even pathologists in terms of information need regarding SSR templates. Nationwide use of the toolbox could provide us with insights into the influencing characteristics of laboratories in order to tailor the toolbox to local information needs. Moreover, Branston et al25  organized both educational and training sessions on SSR. This is in line with the requests of the pathologists interviewed in our study. However, digital education was preferred instead of in-class education, and the facultative character of our e-learning was highly valued. Despite these positive aspects of its feasibility, the long-term effect of e-learning on professional behavior in health care is not yet clear.44  Therefore, future research should evaluate the effectiveness of e-learning on the use of SSR among pathologists.

In our study, audit and feedback on local SSR usage was solely provided verbally to the contact person of the test laboratory. However, a Cochrane review on the effectiveness of audit and feedback strategy elements mentions the importance of having both verbal and written formats.45  Another study discussing 15 suggestions for designers of practice feedback states the importance of constructing feedback through social interaction, which will become easier when it is provided in written formats.46  Therefore, the provision of audit and feedback needs to be adapted to a more shareable format, challenging individual pathologists to increase their use of SSR in daily practice.

Studies of Renshaw et al4749  showed the impact of the method of report creation, content of SSR items, and format of these items on the accuracy of the pathology report and therefore patient care. In both studies on SSR-related strategy elements, discussions took place regarding SSR content.25,27  Moreover, although stated as the ultimate goal by Nakleh et al50  in 2017, a recent published article on the electronic cancer checklists of the College of American Pathologists shows no clear process on development and revisions of these checklists.51  In our study, the need for proper discussions on SSR content was clearly stated as the improvement with the most potential by the pathologists, and the availability of the “Feedback Button” was not deemed sufficient to support pathologists in their improvement of content of SSR templates. Because of the tremendous importance of these content features on SSR implementation, stated by pathologists in this study and in other studies, future research should be focused on the expansion of a widely supported SSR development and revision strategy and evaluate its effectiveness and feasibility on a nationwide scale. Part of this strategy could be aimed at evaluating current SSR output and testing the use of modifiers to quantitative values, holding a promise to improve SSR accuracy.51,52 

The strengths of this study are the stepwise approach of selecting, developing, and testing a toolbox tailored to a previous analysis of barriers and facilitators and the additional step of testing this toolbox in multiple laboratories before introducing the toolbox on a nationwide scale. Moreover, by conducting the expert meeting to determine implementation strategy element selection and interviews to determine the feasibility of the strategy elements, the end users were actively involved in the selection, development, and testing phases of the tailored strategy. Others involved in the implementation of SSR may use the results of this study in their own process of incorporating SSR into daily pathology practice.

This study does have some limitations. First, the feasibility study was conducted during a nationwide lockdown period (coronavirus disease 2019). As a consequence, the number of pathology reports decreased in the after-measurement period, in line with the decrease in oncologic diagnosis in this period.53  Furthermore, this crisis may have negatively impacted the communication between pathologists about the toolbox. However, by designing digitally accessible strategy elements and having monthly telephone conversations, we were able to continue the study during this period and to monitor changes in the working environment. Second, this study showed a significant stronger increasing trend in SSR usage among test laboratories compared to control laboratories. We were able to test the differences in change statistically, but we have to interpret this result with caution, because the used study design does not exclude other reasons for this change. In a subsequent next study in which the implementation strategy elements will be available for all Dutch pathology laboratories, effectiveness might be determined by an interrupted time-series analysis. Last, improving the implementation of SSR demands an investment, both monetary and in man-hours, from all involved stakeholders. However, implementation costs are negligible compared to losses due to suboptimal communication and treatment resulting from incomplete reporting.54 

Six implementation strategy elements were selected and developed, incorporated within an evidence-based multifaceted strategy, and combined in a toolbox, and their effectiveness and feasibility were tested to improve the implementation of SSR in oncologic pathology. Results on effectiveness showed a significant stronger increasing trend in SSR usage among test laboratories. Regarding feasibility, e-learning and the approachable feedback method were found to be most feasible and effective, owing to their additional value to current SSR practices. However, familiarity with all implementation strategy elements could be improved by using national dissemination strategies. Therefore, the next step is to optimize the toolbox and to share the toolbox nationwide and, thereafter, to evaluate both the effectiveness and feasibility of these strategy elements on a national level.

We thank the experts of the nationwide expert panel for their involvement in the consensus meeting. We thank the 6 laboratories and the contact persons for their contribution to this study. In particular, we thank the pathologists who participated in the interview and survey study. We thank Angelique Schlief, BSc, for her assistance regarding data collection and data analysis and Reinier Akkermans, MSc, for his assistance regarding data analysis.

1.
Strobel
SL.
Enhancing the pathologist's role at hospital tumor boards
.
Ann Clin Lab Sci
.
2006
;
36
(3)
:
243
247
.
2.
Hewer
E.
The oncologist's guide to synoptic reporting: a primer
.
Oncology
.
2009
;
98
(6)
:
396
402
.
3.
Schwartz
LH,
Panicek
DM,
Berk
AR,
Li
Y,
Hricak
H.
Improving communication of diagnostic radiology findings through structured reporting
.
Radiology
.
2011
;
260
(1)
:
174
181
.
4.
Chambers
AJ,
Pasieka
JL,
Temple
WJ.
Improvement in the accuracy of reporting key prognostic and anatomic findings during thyroidectomy by using a novel Web-based synoptic operative reporting system
.
Surgery
.
2009
;
146
(6)
:
1090
1098
.
5.
Samartine
S,
White
L,
McKeon
D,
Becker
M.
Enhancing structured reporting: improving quality by tailoring the report to the clinical scenario
.
J Am Coll Radiol
.
2015
;
12
(8)
:
845
847
.
6.
Kahn
CE
Jr,
Heilbrun
ME,
Applegate
KE.
From guidelines to practice: how reporting templates promote the use of radiology practice guidelines
.
J Am Coll Radiol
.
2013
;
10
(4)
:
268
273
.
7.
Larson
DB,
Towbin
AJ,
Pryor
RM,
Donnelly
LF.
Improving consistency in radiology reporting through the use of department-wide standardized structured reporting
.
Radiology
.
2013
;
267
(1)
:
240
250
.
8.
Kennedy
ED,
Milot
L,
Fruitman
M,
et al
Development and implementation of a synoptic MRI report for preoperative staging of rectal cancer on a population-based level
.
Dis Colon Rectum
.
2014
;
57
(6)
:
700
708
.
9.
Dhillon
RS,
King
JA,
Goldschlager
T,
Wang
YY.
Synoptic reporting of pituitary magnetic resonance imaging
.
ANZ J Surg
.
2018
;
88
(5)
:
460
463
.
10.
Bretthauer
M,
Aabakken
L,
Dekker
E,
et al
Reporting systems in gastrointestinal endoscopy: requirements and standards facilitating quality improvement: European Society of Gastrointestinal Endoscopy position statement
.
United European Gastroenterol J
.
2016
;
4
(2)
:
172
176
.
11.
Hadlock
SD,
Liu
N,
Bernstein
M,
et al
The quality of colonoscopy reporting in usual practice: are endoscopists reporting key data elements?
Can J Gastroenterol Hepatol
.
2016
;
2016
:
1929361
.
12.
Maniar
RL,
Hochman
DJ,
Wirtzfeld
DA,
et al
Documentation of quality of care data for colon cancer surgery: comparison of synoptic and dictated operative reports
.
Ann Surg Oncol
.
2014
;
21
(11)
:
3592
3597
.
13.
Maniar
RL,
Sytnik
P,
Wirtzfeld
DA,
et al
Synoptic operative reports enhance documentation of best practices for rectal cancer
.
J Surg Oncol
.
2015
;
112
(5)
:
555
560
.
14.
Schneider
L,
Shargall
Y,
Schieman
C,
et al
Design of a consensus-derived synoptic operative report for lung cancer surgery
.
Ann Thorac Surg
.
2014
;
97
(4)
:
1163
1168
.
15.
Temple
WJ,
Chin-Lenn
L,
Mack
LA,
Alberta
CS.
Evaluating population-based breast cancer surgical practice in real time with a web-based synoptic operative reporting system
.
Am J Surg
.
2014
;
207
(5)
:
693
697
.
16.
Sluijter
CE,
van Lonkhuijzen
LR,
van Slooten
HJ,
Nagtegaal
ID,
Overbeek
LI.
The effects of implementing synoptic pathology reporting in cancer diagnosis: a systematic review
.
Virchows Arch
.
2016
;
468
(6)
:
639
649
.
17.
International Collaboration on Cancer Reporting.
International Collaboration on Cancer Reporting Web site
.
September
5,
2019
.
18.
Ellis
DW,
Srigley
J.
Does standardised structured reporting contribute to quality in diagnostic pathology: the importance of evidence-based datasets
.
Virchows Arch
.
2016
;
468
(1)
:
51
59
.
19.
Sluijter
CE,
van Workum
F,
Wiggers
T,
et al
Improvement of care in patients with colorectal cancer: influence of the introduction of standardized structured reporting for pathology
.
JCO Clin Cancer Inform
.
2019
;
3
:
1
12
.
20.
Baranov
NS,
Nagtegaal
ID,
van Grieken
NC,
et al
Synoptic reporting increases quality of upper gastrointestinal cancer pathology reports
.
Virchows Arch
.
2019
;
475
(2)
:
255
259
.
21.
Netherlands Comprehensive Cancer Organisation.
OncolinQ Web site
.
2019
.
22.
International Collaboration on Cancer Reporting.
Datasets 2020 Web site
.
April
16,
2020
.
23.
College of American Pathologists.
Protocols and guidelines Web site
.
2020
.
24.
Haugland
HK,
Casati
B,
Dørum
LM,
Bjugn
R.
Template reporting matters—a nationwide study on histopathology reporting on colorectal carcinoma resections
.
Hum Pathol
.
2011
;
42
(1)
:
36
40
.
25.
Branston
LK,
Greening
S,
Newcombe
RG,
et al
The implementation of guidelines and computerised forms improves the completeness of cancer pathology reporting—the CROPS project: a randomised controlled trial in pathology
.
Eur J Cancer
.
2002
;
38
(6)
:
764
772
.
26.
PALGA foundation.
Annual reports 2020 Web site
.
November
12,
2020
.
27.
Srigley
JR,
McGowan
T,
Maclean
A,
et al
Standardized synoptic cancer pathology reporting: a population-based approach
.
J Surg Oncol
.
2009
;
99
(8)
:
517
524
.
28.
Casati
B,
Bjugn
R.
Structured electronic template for histopathology reporting on colorectal carcinoma resections: five-year follow-up shows sustainable long-term quality improvement
.
Arch Pathol Lab Med
.
2012
;
136
(6)
:
652
656
.
29.
Hassell
LA,
Parwani
AV,
Weiss
L,
Jones
MA,
Ye
J.
Challenges and opportunities in the adoption of College of American Pathologists checklists in electronic format: perspectives and experience of Reporting Pathology Protocols Project (RPP2) participant laboratories
.
Arch Pathol Lab Med
.
2010
;
134
(8)
:
1152
1159
.
30.
Flottorp
SA,
Oxman
AD,
Krause
J,
et al
A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice
.
Implement Sci
.
2013
;
8
:
35
.
31.
Grol
R,
Wensing
M.
What drives change: barriers to and incentives for achieving evidence-based practice
.
Med J Aust
.
2004
;
180
(6 suppl)
:
S57
S60
.
32.
Swillens
JEM,
Sluijter
CE,
Overbeek
LI,
Nagtegaal
ID,
Hermens
RPMG.
Identification of barriers and facilitators in nationwide implementation of standardized structured reporting in pathology: a mixed method study
.
Virchows Arch
.
2019
;
475
(5)
:
551
561
.
33.
Swillens
JEM,
Voorham
QJM,
Nagtegaal
ID,
Hermens
RPMG.
Improving interdisciplinary communication
:
barriers and facilitators for implementation of standardized structured reporting in oncology [published online October 15
,
2021]
.
J Natl Compr Canc Netw. doi:10.6004/jnccn.2021.7002
34.
Grol
R,
Grimshaw
J.
Evidence-based implementation of evidence-based medicine
.
Jt Comm J Qual Improv
.
1999
;
25
(10)
:
503
513
.
35.
Powell
BJ,
Waltz
TJ,
Chinman
MJ,
et al
A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project
.
Implement Sci
.
2015
;
10
(1)
:
21
.
36.
Taxonomy
EPOC. EPOC
2015 Web site. epoc.cochrane.org/epoc-taxonomy. Accessed,
March
25,
2020
.
37.
Foundation
PALGA.
PALGA protocols Web site
.
2019
.
38.
Wensing
M,
Oxman
A,
Baker
R,
et al
Tailored implementation for chronic diseases (TICD): a project protocol
.
Implement Sci
.
2011
;
6
(1)
:
103
.
39.
Cuble.
Online training
.
June
15,
2019
.
40.
Van Teijlingen
E,
Hundley
V.
The importance of pilot studies
.
Nurs Stand
.
2002
;
16
(40)
:
33
.
41.
Hulscher
ME,
Laurant
MG,
Grol
RP.
Process evaluation on quality improvement interventions
.
Qual Saf Health Care
.
2003
;
12
(1)
:
40
46
.
42.
Bowen
DJ,
Kreuter
M,
Spring
B,
et al
How we design feasibility studies
.
Am J Prev Med
.
2009
;
36
(5)
:
452
457
.
43.
Mayer
RE,
Moreno
R.
A cognitive theory of multimedia learning: implications for design principles
.
J Educ Psychol
.
1998
;
91
(2)
:
358
368
.
44.
Sinclair
PM,
Kable
A,
Levett-Jones
T,
Booth
D.
The effectiveness of Internet-based e-learning on clinician behaviour and patient outcomes: a systematic review
.
Int J Nurs Stud
.
(2016)
;
57
:
70
81
.
45.
Ivers
N,
Jamtvedt
G,
Flottorp
S,
et al
Audit and feedback: effects on professional practice and healthcare outcomes
.
Cochrane Database Syst Rev.
2012
;
(6):CD000259.
46.
Brehaut
JC,
Colquhoun
HL,
Eva
KW,
et al
Practice feedback interventions: 15 suggestions for optimizing effectiveness
.
Ann Intern Med
.
2016
;
164
(6)
:
435
441
.
47.
Renshaw
AA,
Mena-Allauca
M,
Gould
EW,
Sirintrapun
SJ.
Synoptic reporting: evidence based review and future directions
.
JCO Clin Cancer Inform
.
2018
;
2
:
1
9
.
48.
Renshaw
MA,
Renshaw
SA,
Mena-Allauca
M,
et al
Performance of a web based method for generating synoptic reports
.
J Pathol Inform
.
2017
;
8
(10)
:
13
.
49.
Renshaw
AA,
Gatcliffe
T,
Gould
EW.
Synoptic report response options directly impact patient care
.
Arch Pathol Lab Med
.
2020
;
144
(8)
:
918
919
.
50.
Nakhleh
RE,
Fitzgibbons
PL,
Washington
MK,
Baker
TP,
Berman
MA,
Cagle
PT.
The future of College of American Pathologists cancer protocols: maintaining a commitment to patient safety while improving the user experience
.
Arch Pathol Lab Med
.
2017
;
141
(9)
:
1153
1154
.
51.
Torous
VF,
Allan
RW,
Balani
J,
et al
Exploring the College of American Pathologists electronic cancer checklists: what they are and what they can do for you
.
Arch Pathol Lab Med
.
2021
;
145
(4)
:
392
398
.
52.
Renshaw
AA,
Gould
EW.
Improving reporting of tumor size in synoptic reports
.
Arch Pathol Lab Med
.
2021
;
145
(8)
:
969
972
.
53.
Dinmohamed
AG,
Visser
O,
Verhoeven
RH,
et al
Fewer cancer diagnoses during the COVID-19 epidemic in the Netherlands
.
Lancet Oncol
.
2020
;
21
(6)
:
750
751
.
54.
Wensing
M,
Grol
R.
Implementation of change in healthcare: a complex problem
.
In:
Wensing
M,
Grol
R,
Grimshaw
J,
eds.
Improving Patient Care: The Implementation of Change in Health Care. 3rd ed
.
Hoboken, NJ
:
Wiley-Blackwell;
2020
:
3
17
.
55.
Proctor
EK,
Powell
BJ,
McMillen
JC.
Implementation strategies: recommendations for specifying and reporting
.
Implement Sci
.
2013
;
8
(1)
:
1
11
.
56.
Kok
G,
Gottlieb
NH,
Peters
GJY,
et al
A taxonomy of behaviour change methods: an intervention mapping approach
.
Health Psychol Rev
.
2016
;
10
(3)
:
297
312
.

Author notes

This study received funding from the Dutch Cancer Society (grant No. 8281).

Supplemental digital content is available for this article at https://meridian.allenpress.com/aplm in the December 2022 table of contents.

Competing Interests

The authors have no relevant financial interest in the products or companies described in this article.

Presented as a poster presentation at the American Society of Clinical Oncology Quality Care Symposium 2019; September 6–7, 2019; San Diego, California; in part as an oral presentation at the 32nd Congress of the European Society of Pathology and XXXIII International Congress of the International Academy of Pathology; December 8, 2020; Glasgow, United Kingdom (digital); and in part as an oral presentation at the 16th annual Guidelines International Network conference; October 25, 2021 (digital).

Supplementary data