Context.—Text reports convey critical medical information from pathologists, radiologists, and subspecialty consultants. These reports must be clear and comprehensible to avoid medical errors. Pathologists have paid much attention to report completeness but have ignored the corresponding issue of report comprehension. This situation presents an increasingly serious potential problem. As laboratories are consolidated and as reports are disseminated in new ways (eg, via the World Wide Web), the target audience becomes more diverse and less likely to have any contact with pathologists beyond the written reports themselves.
Objective.—To compare clinician comprehension with pathologist intent in written pathology reports.
Methods.—Typical surgical pathology reports relevant to surgeons and covering a range of specimen complexity were taken from our hospital files. Questionnaires based on these cases were administered open-book-examination style to surgical attending physicians and trainees during surgical conferences at an academic medical center.
Main Outcome Measures.—Scores from questionnaires.
Results.—Surgeons misunderstood pathologists' reports 30% of the time. Surgical experience reduced but did not eliminate the problem. Streamlined report formatting exacerbated the problem.
Conclusions.—A communication gap exists between pathologists and surgeons. Familiarity with report format and clinical experience help reduce this gap. Paradoxically, stylistic improvements to report formatting can interfere with comprehension and increase the number of misunderstandings. Further investigation is required to reduce the number of misunderstandings and, thus, medical errors.
Just as medical language may be hard for lay people to understand, medical specialists may use language that is obscure to practitioners outside their specialty. Among specialists, the language of diagnostic anatomic pathologists is arguably furthest from daily medical discourse. There has been an implicit recognition of this fact in efforts to standardize pathology reporting. These efforts have lead to the adoption of the so-called Bethesda system for Papanicolaou smear reporting1 and similar attempts to standardize reports in other areas, including tumor resections, transplant biopsies, and liver biopsies.2–4 However, little explicit attention has been focused on the ability of clinicians to understand pathology reports, despite the role of reports in guiding treatment.
Advances in anatomic pathology information systems offer an opportunity to change pathology reports, for example, by reporting via the World Wide Web and digital imaging. We hoped to improve report comprehension by using new forms of presentation,5–9 beginning with a streamlined text report format, and we hoped to show empirically that our new formats increased comprehension or reduced errors. Contrary to our original hypothesis, our modified anatomic pathology report formats reduced comprehension when compared to our hospital system's standard report. There were a surprising number of misunderstandings of what pathologists intended to convey, even when using our system's original report format. More problems are likely, given current trends to present reports solely on computer terminals as part of a computer-based patient record. New report formats may decrease comprehension, presumably due to reduced familiarity. The problem of report comprehension warrants exploration.
We adopted an experimental design similar to Fletcher's10 approach for testing problem-oriented medical record comprehension. Six anatomic pathology reports were selected to cover a range of specimens and complexity. They included 1 report each of a transurethral resection of a bladder tumor, a medical liver biopsy, a transplant kidney biopsy (using the Banff11 reporting scheme), a parathyroid resection, a transurethral resection of prostate, and a gastric biopsy. Report text came from our hospital's online anatomical pathology system (CoPath, Dynamic Healthcare Systems, Waltham, Mass). We altered names and identifying information to protect confidentiality.
From each original report (Figure 1) we created a new streamlined version (Figure 2). Key pathologic findings appeared immediately after the patient's name. Nonclinical institutional information was placed at the bottom of the report. Compact Bell Centennial printer font12,13 permitted us to present the name and essential findings in a larger type size, while keeping specimen preparation details legible. We changed the text of each report only as grammatically necessary for this new format. To control for changes in type style and page arrangement, we also created a partially modernized version of each original (Figure 3) with the original text set in Bell Centennial font and with final findings moved higher on the page.
We prepared a short clinical scenario for each case, then we prepared questions to be answered after reading a version of the report. Intended answers to these questions (yes, no, or not stated) were determined by 2 attending pathologists (J.C. and R.J.H.) after reviewing the reports. There was agreement between the pathologists in all cases. Figure 4 shows questions for the report appearing in Figures 1 through 3. Subjects were allowed 2 chances to answer these questions: first after reading an introduction to the case and the pathology report and setting them both aside, then immediately afterwards in an open-book-examination style with report and questions side by side. All subjects reviewed all 6 cases, but subjects reviewed different combinations of formats (2 cases in each of the 3 formats). Only the more accurate, open-book responses were used in this analysis since charts can be kept open and reports reviewed any number of times during routine clinical practice. Initial review of closed-book answers did not reveal any significantly different trends, just more discrepancies.
Test subjects were 34 general surgical attending physicians and trainees. Testing was scheduled during time volunteered from regularly scheduled team conferences to allow group testing without extending anyone's workday. Average time for completion was 30 minutes. Question books were randomly passed around the room, unmarked and unsigned, except for a code to allow 1 trainee and 1 attending, chosen randomly from among the top 4 scores, to be awarded gift certificates for a local restaurant. Our Human Investigation Committee approved this protocol.
A seventh case was prepared and used as a sample for explaining the test procedure. It was not counted in any of the results.
Table 1 shows a disconcerting discordance between pathologists' intended meanings and interpretation by surgeons; the crude discrepancy rate was 30%. As expected, experience improved performance, although the discrepancy rate was still significant for both attending physicians and senior trainees reading reports in their original format (Table 1; Figure 5). All data tally responses with reports printed and open right next to the questionnaires. We did not conclude that our subjects were wrong, only that the pathologists' intended meanings were not effectively communicated. Discrepancy rates from Table 1 may unfairly magnify possibilities for clinical misadventure. In real hospital practice, surgeons can ask pathologists to clarify ambiguities or resolve unanswered questions if the surgeons recognize their clinical uncertainty. Unfortunately, we found no reliable correlation between discordance and uncertainty. Figure 6 shows discordant responses plotted against uncertainty ratings. (These are overall case uncertainty ratings. Subjects were asked only to indicate their uncertainty in answering all questions about a case, not in answering individual questions.) A scattering of uncertainty and discordance measurements is seen, and uncertainty is not related in any simple way to inexperience.
As for the effects of changing report format, the larger the change, the larger the discordance rates (Table 1). Our modernized format (Figure 3) yielded higher discordance rates than the original format (Figure 1) overall. Our radically new format (Figure 2) yielded even higher discordance rates. Despite our intended improvements, Table 1 shows that test subjects reading either of our new formats were overall less likely to understand our pathologists' intentions than were those reading reports in the original format.
Since all observed discordance rates seemed unacceptable, we abandoned further analysis related to specific format. We sought, instead, to identify the characteristics of questions—or of reports in general—that lead to discrepancies. We divided questions into the following 3 categories: those which can be answered (1) directly from the text of a report, (2) after some interpretation of a report, or (3) only with a higher level of (clinical) understanding or terminology, including the operation of a pathology laboratory. Table 2 lists abbreviated questions in order of increasing discordance. In general, the more interpretation required, the more discrepancies occurred.
Questions that were answered by an explicit statement in the diagnosis section of the report were usually answered correctly (eg, question 4·2, presence or absence of Helicobacter pylori in a gastric biopsy). Surprisingly, question 1·2, which required examination of the gross description section of the report, was always answered correctly. (Perhaps “40 grams” stood out as this report's only numeric finding.) Questions concerning the presence or absence of carcinoma were virtually always answered correctly. Errors in this category were limited to 1 surgical intern, who may have just finished night call,14–16 and 2 attending physicians. The 2 attending physicians actually answered “not stated,” and under normal circumstances could have asked for clarification.
Discrepancies increased in cases with multipart specimens, even if our questions were answered explicitly in a report's diagnosis section (questions 5·1 and 6·5). Pathologists generally note only the presence of certain tissues in the specimen; presumably, readers understand that tissues not mentioned were not found. Our clinician readers appeared not to share this convention (questions 6·4 and 6·2). There were similar misunderstandings about specimen adequacy. In 2 such questions (questions 2·1 and 5·4), 60% of clinicians did not recognize a specimen to be adequate even when a specific diagnosis was made by the pathologist. The Banff system for kidney biopsy reports explicitly states that diagnosis should not be made in the absence of adequate tissue.
Two often misunderstood questions required a knowledge of how pathology laboratories function (questions 1·3 and 1·4). In the first case, full-thickness bowel was found on the slide but was stated to be a contaminant. Many clinicians interpreted this to mean that there was a procedural complication. In the second case, the report stated that only representative prostate chips were examined histologically, but clinicians failed to register this. Both cases have medicolegal implications.
Other questions with medicolegal implications included mistaken impression of a frozen section–final diagnosis difference and mistaken impression of having removed thymus tissue (questions 6·1 and 6·4). Our hospital convention is to label each part with the specimen label given by the surgeon. The surgeon labeled the specimen “thymus”; this convention may have left the impression that thymus had in fact been removed.
Another convention at our institution is to not include a histologic description except in unusual or difficult cases. This practice might lead to higher rates of misunderstanding; however, among the 6 cases studied, more severe problems of comprehension arose when clinicians had to interpret histologic descriptions or understand histologic terminology (questions 2·2, 2·3, 2·4, and 4·5). Histologic descriptions may be of little use to surgeons. For example, question 2·4 assumed a basic knowledge of histologic and ultrastructural characteristics of diabetic glomerulopathy (diabetic patient with a transplanted kidney). Interestingly, this particular case, which was reported in a formal terminology (Banff system), was the least understood overall. A shorthand system such as the Banff system is completely opaque to nonspecialists.
Subjects were solicited for comments after each session. No subject suggested that the questions or test format was clinically unfair. A number of our surgical oncology team members did ask that a structured report format be added to considerations for new computer reports. A number of surgeons indicated a preference for “the darker printing” of the modernized reports.
An overall discordance rate of 30% for surgeons and surgical trainees answering open-book questions about anatomic pathology reports was surprising. It is reassuring that more experienced attending physicians performed better than less experienced trainees, and that presence or absence of invasive cancer was virtually always recognized. However, the discrepancies that remained were not just about minor findings. Reading a kidney transplant biopsy report, 13 (38%) of 34 clinicians had the mistaken impression of no acute rejection. With respect to a bladder biopsy report, there was widespread failure to recognize deeply invasive bladder cancer and carcinoma in situ, both of which are key features in determining therapy.
Our study took place in an optimal setting with printed paper reports and a block of time set aside from hospital ward demands. In clinical practice, reports may be seen only on a computer screen in a busy, noisy ward. Our study shows that on average trainees understand reports less well than attending surgeons. Unfortunately, in academic settings, reports may be retrieved by residents, to be reported back to attending physicians. Nurse-practitioners or physician assistants may screen pathology reports prior to physician review in private practice settings (Priscilla Jencks, MPH, MSN, personal communication, June 3, 1998). While we have not studied these midlevel practitioners, similar difficulties in comprehension are likely, as their formal training in pathology is minimal or nonexistent.
Most of our subjects had been reading reports in our original format for years. Changing the report format may have caused a negative short-term effect on these readers (ie, report disorientation). A long-term study might show improvement above the starting baseline. Students and subjects in their first postgraduate year, who were less familiar with any report format, demonstrated less degradation with new formats (Table 1). We suspect it takes a year to establish short-term report preference based on format. Based on these data, the new formats may ultimately be shown to be superior. Our improvements in report format comprised changes to type font, wording, and page layout (position). Perhaps these factors must be varied and tested separately.
Other factors confound our findings. It is possible our test setting eliminated critical clinical motivation for report interpretation. Moreover, we have no way of knowing which discrepancies in interpretation would lead to real medical errors. A prospective study of report interpretation in clinical settings can be imagined, although it would be difficult to execute. Such an investigation would require separation of clinical error from poor outcome, and its results would likely be confounded by the observers themselves (Hawthorne effect).
Essentially no empirical literature exists on pathology report comprehension to provide a guide for improvement. Professional societies and authors have recommended various styles for clinical reports.2–4 It is unknown if any of the proposed standardized reports actually improve understanding by clinicians. However, standardized reports encourage explicit reporting, and our findings suggest that the less interpretation by readers the better. Adopting standardized systems comes at a price: new nomenclatures can lead to balkanization of knowledge. Standardized systems are not a panacea. They codify presentation of common findings in routine clinical situations, but cannot reflect the wide range of findings commonly encountered in clinical practice. Our problematic cases could not have been reported in any standardized way.
Results obtained here may or may not apply to other consultation reports (eg, radiology or psychiatry). Do primary care physicians understand reports from subspecialists to whom they refer patients for diagnosis and treatment recommendations? Based on their understanding or misunderstanding, primary clinicians often implement and follow up treatment on their own, because of managed care constraints. Leape17 did not focus on professional communication as a source of medical errors in his review in 1994. More recently, Bhasale and colleagues18 described poor communication between health professionals as a contributing factor in 19 of 100 incidents registered by Australian general practitioners. Their findings reinforce the idea that communication failure is an important source of medical error.19
Technical writers have a similar challenge when conveying commercial or industrial information.20,21 Unfortunately, it is not clear how guidelines for instructing a homeowner to assemble a lawnmower, for example, should be applied to diagnostic reports for trained professionals. New information systems allow inclusion of line drawings, gross photographs, and photomicrographs in reports. Other purely stylistic improvements also are possible, for example, placing important statements up front or using a more legible type font. However, our study demonstrates that stylistic improvements alone may reduce rather than increase comprehension. Clinicians and pathologists need to improve medical communication for patient care, while avoiding the disruption of established familiar report formats currently protecting clinical content.
We thank the members of the Department of Surgery, Yale University School of Medicine, New Haven, Conn, for their time and cooperation in this study. We also appreciate an initial review and help coordinating test sessions provided by Dana Andersen, MD. Elizabeth C. Yen, JD, was of great assistance assembling test books. We also thank the late Robert Byck, MD, for his thoughtful critique of our experimental design and early drafts of our manuscript.