Standardized Synoptic Cancer Pathology Reports — So What and Who Cares?: A Population-Based Satisfaction Survey of 970 Pathologists, Surgeons, and Oncologists

A quasi-experimental, cross-sectional, descriptive design was used. The focus of the program evaluation was to determine the impact of implementation of standardized synoptic pathology reporting on physician satisfaction in areas such as process (eg, timeliness of reports, completeness, need for follow-up) and practice (eg, information to support clinical decision making).

The inclusion criterion for this evaluation study was for a hospital to have implemented synoptic pathology reporting by March 31, 2010, resulting in a total of 33 hospitals available for inclusion in the study. The primary targeted population for the study included pathologists, as those responsible for the generation of pathology reports, and treating physicians (ie, surgeons, medical oncologists, and radiation oncologists) as the end users of the pathology reports. Because the initial phase of synoptic reporting was focused on 5 of the College of American Pathologists checklists (eg, lung, breast, colorectal, prostate, and endometrium), physicians involved in treating these 5 cancers were the primary focus. Contact information (e-mail and or fax numbers) for the identified stakeholder groups was requested and obtained from 27 of the 33 identified hospitals. This resulted in a final sample consisting of a total of 970 physicians, representing pathologists (n = 252), surgeons (n = 462), medical oncologists (n = 128), and radiation oncologists (n = 128).

The survey items were developed with a desire to obtain information related to physician perceptions of timeliness, completeness, usability, and accuracy of the reports as well as overall user satisfaction. Two surveys were developed, with one targeted for pathologists and the other for treating physicians (ie, surgeons, medical oncologists, and radiation oncologists). The items on each survey were consistent on both versions, with the exception of one item on each version that was specific to each stakeholder group (ie, pathologists and treating physicians). The initial items were created by members of the evaluation team and were then reviewed for face and content validity by 8 content experts, including members of the pathology reporting team, radiation oncologists, medical oncologists, and surgeons. The resulting 11-item survey asked participants to answer the items by comparing standardized synoptic reporting with narrative reporting methods using a 5-point Likert scale (1 = significantly less than narrative reports, 3 = the same as narrative reports, 5 = significantly better than narrative reports). The survey was available in a Web-based or hard-copy format. To enhance response rates, all participants who completed the survey were provided the option to enter their name for 1 of 4 cash drawings of $1000 Canadian. A systematic review regarding methods for improving physician response rates¹⁸  identified that survey length, mode (Web-based, mail, or phone), and the frequency of reminders were key factors to consider when developing a survey process involving physicians. The use of financial incentives was also shown to increase response rates.¹⁹  As this evaluation was undertaken for the purposes of program review quality assurance, ethics review was not required, as stipulated under Article 2.5 of the Canadian Tri-Council Policy Statement on the Ethical Conduct for Research Involving Humans.²⁰  Confidentiality of individual responses was assured. Participants had the option of providing their name only if they wished to be included in the cash drawing and/or if they did not wish to receive future reminders or communications regarding the survey.

Data collection followed the Tailored Design Method as described by Dillman.²¹  For physicians with e-mail contact information, all communication regarding the survey was sent via e-mail, which included the hyperlink to the Web-based survey. For those without e-mail, all communications, including the survey, were sent by fax to the clinicians' office. A secure fax number was provided for the return of completed paper surveys. Reminder notices were sent out every 2 weeks and targeted only at those who had not yet returned the survey (based on the names provided on returned surveys as described above). Survey results were entered in SPSS version 16.0 (SPSS Inc., Chicago, Illinois) for data analysis.

A total of 498 surveys were completed, representing a 51% response rate. The response rate by specialty ranged from 39% (surgeons) to 68% (pathologists), with 45% and 55% response rates from medical oncologists and radiation oncologists, respectively. The respondents were from a variety of practice settings, with the majority (41%) indicating their primary practice setting as a teaching hospital affiliated with a regional cancer center. Forty-three percent indicated they had 10 to 20 years of experience in practice, followed by those with 21 to 30 years (27%) and those with less than 10 years (25%). When asked to describe the average number of cancer resection pathology reports completed each month, the majority of pathologist respondents (43%) indicated they completed between 10 and 25 reports, on average, a month. This was similar to the amount of pathology reports reviewed by the majority of clinicians (43%). Prior to the implementation of standardized synoptic reporting, the most common method of pathology reporting used was narrative reporting (74%), with 19% indicating the use of some form of electronic or synoptic-like pathology reporting in place. See Table 1 for respondent demographics.

Results of descriptive statistics revealed that based on a 5-point Likert scale, with 5 indicating synoptic reports are significantly better than narrative reports, the vast majority of physicians who responded reported that the standardized synoptic pathology reports were significantly better than narrative reports for all items, with mean scores ranging from 3.84 to 4.77. Table 2 provides the mean and standard deviation scores for survey items based on the responses for pathologists and treating physicians (eg, surgeons, medical oncologists, and radiation oncologists), with “na” indicating questions that were not included on the survey and therefore did not have responses for the particular stakeholder group.

Regarding the amount of time to produce and receive pathology reports, participants were again provided with a 5-point Likert scale (1 = significantly less than narrative reports, 3 = about the same, and 5 = significantly more than narrative reports). Pathologists reported that the time to produce the reports was more than that of narrative reports (mean, 3.51; SD = 1.43), and similarly, treating physicians also reported a slight increase in time to receive the reports (mean, 3.41; SD = 1.16). The survey provided the option of then indicating the approximate percentage of time (eg, more or less) required. To further understand the impact of synoptic reporting on perceptions of work flow, a dichotomous variable was created and responses recoded based on more or less response options. Results indicated that those pathologists who indicated synoptic pathology reporting required more time reported that synoptic pathology reporting required 25% to 50% more time to complete.

Qualitative comments that accompanied these responses indicated that the major contributor to the increase in time was associated with technology-related factors such as software glitches, upgrades, and information system–related issues. For the treating physician group, the vast majority (60%) reported that the amount of time required to obtain the final pathology report was “about the same as” narrative reports. So, although the pathologists reported that synoptic reports took longer than narrative reports, the end users of the reports did not perceive a difference in the time required to obtain the report.

In terms of overall satisfaction, both groups reported synoptic pathology reports as being better than narrative reports, with treating physicians reporting higher levels of satisfaction with the overall process (mean, 4.52; SD = 0.991) and the level of information provided (mean, 4.85; SD = 0.901) as compared with pathologists' satisfaction with the process (mean, 4.08; SD = 1.34) and level of information provided (mean, 4.08; SD = 1.44).

In order to determine the factors that may have contributed to satisfaction, a correlation analysis was conducted. Results showed a moderately strong positive relationship between respondents' perceptions of overall satisfaction with the level of information provided in synoptic reports and respondents' perceptions of the completeness of the reports for clinical decision making (r = 0.750, n = 313, P < .001), comparison with accepted content standards (r = 0.692, n = 313, P < .001), ease of finding information for clinical decision making (clinicians: r = 0.663, n = 314, P = .001; pathologists: r = 0.510, n = 171, P < .001), and the report's ability to facilitate a consistent approach to diagnostic and prognostic factors (clinicians: r = 0.717, n = 312, P < .001; pathologists: r = 0.638, n = 168, P < .001) (see Table 3).

Dependent t tests were conducted to compare the differences in the mean scores of pathologists' and treating physicians' perceptions of overall satisfaction, indicating a statistically significant difference in scores for overall satisfaction with the synoptic reporting process (t₁₆₉ = 3.044, P = .003). This result is consistent with the frequency distribution of pathologists' responses, with greater variation in responses than their physician counterparts. As the implementation of synoptic pathology reporting had a direct impact on pathologist practice and work flow, it is not surprising that pathologists' overall level of satisfaction would be different from that of their colleagues.

Analysis of variance was conducted to compare mean satisfaction scores for pathologists, surgeons, and medical oncologists, based on demographic variables such as practice setting, years of experience, number of pathology reports, and/or previous method of pathology reporting prior to implementation of synoptic pathology reporting, with no statistically significant differences found in the overall satisfaction scores based on any of the demographic variables.

The qualitative data were thematically summarized using conventional content analysis methodology.²²  Upon review of the qualitative comments provided, 2 main themes emerged: practice-related and process-related.

Practice-related comments referred to the improvement of the information available (eg, “allows me to find the information I need quickly and efficiently;” “by having a common language, this improves the efficiency of patient management”) and error reduction (eg, “synoptic format reduces the chance for error or forgetting to include a specific parameter that is significant for a cancer case”). Practice-related issues also identified concern the assumption that completeness equates to accuracy (eg, “the main issue with synoptic reporting is that completeness can mask accuracy”; “synoptic reports are a form of presentation but do not necessarily mean that the data is more accurate”).

Process-related comments were centered on technology and usability of the reports because of software requirements/restrictions (eg, “less convenient mostly due to software formatting limitations”; “the rigidity of the choices do not necessarily reflect the complexity of the case we examine”).

The adoption of a common electronic reporting standard by pathologists and hospitals in Ontario is unprecedented for a jurisdiction of this size and complexity. The achievement of 92% of hospitals reporting at level 6 and a 94% completeness rate against the College of American Pathologists cancer checklist standard is quite significant when compared with a recent study using College of American Pathologists Q-Probes data that indicated that the completeness rate, in the 86 American institutions included in the study, was 68%.¹  As is evident in the results of this study, physician satisfaction with standardized pathology reports is high when the information required to support diagnostic and prognostic decision making is readily available, relevant, and timely. Technologic considerations in the design and usability of standardized reports are vital to enabling access to information of importance to clinicians and minimizing the impacts on work flow and workload.

The authors would like to acknowledge Jody Whitfield, research assistant, for her invaluable assistance with the data collection phase of this evaluation study.