Challenges for Data Extraction From Narrative Pathology Reports

To the Editor.—In the report “Do Synoptic Reports Add Value in Prostate Needle Biopsies?” Drs Renshaw and Gould¹  described a software parsing method to extract diagnostic data from the pathology report of 100 prostatic core biopsies and concluded that “prostate needle biopsy reports are easily amenable to structured data extraction without the need for a separate synoptic report.”

Their conclusion, however, rests almost exclusively on 2 conditions: (1) uniform tissue sampling mode (ie, one core per biopsy) and (2) simple dichotomy of diagnosis (ie, prostatic adenocarcinoma versus other). First, as we know, tissue sampling mode largely depends on the urologists who obtain the biopsy and frequently, in turn, may depend on the clinical needs. Pathologists do not control this variable. Second, in addition to prostatic adenocarcinoma, other malignancies such as small cell (neuroendocrine) carcinoma, lymphoma, urothelial carcinoma, etc (each accounting for <1% of prostatic malignancy), also occur in the prostate^2,3  but are not graded with the Gleason grade. How does the reported method parse out cases with these tumors when using “Gleason” as the sole identifier? Moreover, the pathology reports frequently reflect the continuum of our diagnostic interpretation, not the presumptuous ideal dichotomy of pathobiology. Examples such as “Findings suspicious for but not diagnostic of (adenocarcinoma)” or “A minute focus of prostatic adenocarcinoma, volume insufficient for grading” do exist but are unlikely to be captured by the described method. More importantly, is the data set without these common elements representative of actual pathology reports?

Far more challenging but quite essential to such a data-extracting project is handling linguistic variable/syntactic variation, especially of the negation statement.^4,5  How does their method segregate “Perineural invasion is present” from “No perineural invasion is identified” or “Negative for perineural invasion?” Although they reported a 100% accuracy of their parsing results, the results should be treated with alarming caution unless a sound validation schema is applied. A rudimentary human-machine cross-validation of a very small cohort (100 biopsies in this case) may be acceptable to test the programming skill, but is not even adequate for a simple proof of concept, let alone for testing the architecture design, programming algorithm, validation strategy, and results.

With these to be clarified, the key question becomes: Could the authors' conclusion hold if their method requires an impractical uniformity of tissue sampling pattern, can only identify one type of tumor in the reports, cannot handle syntactic variation, and is not based on a representative data set?

I do believe that the issue embarked upon by the authors is very important. It's valuable to find a pragmatic solution applicable to reporting core biopsies of other organs/sites as well. Otherwise, the value of such a project would be extremely limited. In other words, the value of such a strategy/method is critically proportionate to its applicability to other data-mining scenarios. Given the importance of the issue and the broad impact of its solution, judicious caution, careful design, and stringent validation must be exercised before reliable, meaningful, and, above all, correct conclusions can be drawn.

The author thanks Mark A. Micale, PhD, for his careful proofread of and constructive input to this commentary.