Validating Laboratory Results in Electronic Health Records: A College of American Pathologists Q-Probes Study

Participants in this CAP Q-Probes study retrospectively reviewed results from 16 different laboratory tests by directly viewing results within the EHR. If the laboratory transmitted results to more than one EHR, the results were reviewed in the EHR primarily used by providers to view laboratory results. Results were reviewed for a spectrum of laboratory tests that were selected to include tests performed by most laboratories, tests often sent to reference laboratories, tests with numeric and textual results, manually entered results, automatically resulted (ie, autoverified/autovalidated), and anatomic pathology reports, including surgical pathology and cytology. These included: creatinine, corrected platelet count, hemoglobin A1c, international normalized ratio, blood bank antibody screen, blood culture with sensitivities, estimated glomerular filtration rate, human immunodeficiency virus (HIV-1) quantitative polymerase chain reaction assay, serum protein electrophoresis, Epstein-Barr virus viral-capsid antigen immunoglobulin (Ig) G antibody, factor V Leiden mutational assay, breast estrogen receptor studies, surgical pathology report with synoptic component, Papanicolaou cytology report with human papillomavirus result, second-trimester maternal (Quad) screen, and heparin-dependent antibody testing. Point-of-care test results and preliminary nonverified results were excluded.

Worksheets were provided to participants to facilitate data collection. One normal and one abnormal (ie, outside of reference range) test result were identified for each test using sources where this information was most readily available (eg, LIS, log books, reference laboratory reports, etc). Recent results were selected whenever possible. Using patient identifiers, the test results were located within the EHR. Surgical pathology and cytology reports were also reviewed in the electronic system used by providers. Because there are often several ways for providers to view laboratory results within an EHR, participants were asked to select the results screen used by most providers. Participants followed their local policies for accessing patient information within the EHR.

Test result accuracy was assessed by comparing the numeric or textual result in the EHR to the result in the LIS, paper worksheets, instruments, or other primary source. Test result completeness was determined by verifying the presence of the following result components: (1) name/address of performing laboratory, (2) linkage of result to the physician of record, (3) date/time of specimen collection, (4) date/time of test result/report, (5) specimen source, (6) units of measurement when applicable, (7) result interpretation when applicable, (8) information regarding condition and disposition of suboptimal specimens (eg, specimen suitability), (9) reference intervals appropriate for patient age/sex when relevant, (10) flagging appropriate to clearly indicate abnormal results (eg, color highlighting, up/down arrows, exclamation marks, etc), (11) audit trail for corrected results, (12) limitations of test, and (13) Food and Drug Administration disclaimer when applicable. Result formatting was assessed by reviewing the result on the EHR computer screen and on a paper printout of the EHR report. The appropriateness of result formatting was judged using the following criteria:

• 1.

  Appropriate: All results, reference information, and other report elements are presented in a visually easily understandable format, with report elements easily located, clearly labeled, and properly aligned.

• 2.

  Appropriate with minor defects: Results are presented with minimal defects. For example, a report with a minor misalignment or formatting problem that does not render the result difficult to read is considered as having a minor defect.

• 3.

  Inappropriate: Results missing one or more reporting elements and/or presented in a way that is difficult for an HCP to interpret, including major misalignment of results and/or inclusion of extraneous information.

Participants were asked to answer survey questions regarding their IT capabilities and practices. Individual associations between the frequency metrics and time intervals with the demographic and practice variables were analyzed using Kruskal-Wallis tests for discrete-valued independent variables and regression analysis for continuous independent variables. Variables with significant associations (P < .10) were then included in a forward-selection multivariate regression model. A significance level of .05 was used for this final model. For the aggregate case results analyses, t tests were used. All analyses were performed using SAS v9.2 (SAS Institute, Cary, North Carolina).

Three major performance indicators were determined for this study, including: (1) the percent of tests accurately transmitted to the EHR, (2) the percent of test results containing essential reporting elements, and (3) the percent of test results transmitted in a usable format (Table 1). Results are based on information provided by 45 participants who submitted test result transmission data for more than 1000 test results in aggregate. These included results that were transmitted to the EHR from an instrument, LIS, paper worksheet, or other primary source. Overall accuracy of result transmission to the EHR was 99.3% (1052 of 1059). There was no difference in the accuracy of result transmission between normal (99.4%; 526 of 529) and abnormal (99.2%; 526 of 530) test results.

There was lower compliance for completeness of test results in that only 69.6% (732 of 1051) of the results reviewed contained all of the essential reporting elements. The overall rate for appropriate formatting of test results was 90.9% (918 of 1010). Appropriately formatted results included those that were correctly configured on the EHR screen views and EHR paper printouts, and did not contain irrelevant or incorrect information. There were 2 practice characteristics that were significantly associated with the major performance indicators (Table 2). First, institutions that had 50% or less of test orders entered electronically by providers had lower test result completeness rates. Second, institutions that rarely received documented complaints regarding the formatting of test results within the EHR had a higher percent of test results that were appropriately formatted.

Participants provided detailed information regarding their transmission methods for tests performed in-house and at reference laboratories (Table 3). Overall, for in-house–performed tests, most laboratories used electronic means to transfer data from the LIS to the EHR (61.8%; 261 of 422 reviewed results) or the data were manually entered into the LIS (33.2%; 140 of 422 results). Results were less commonly transmitted from middleware to the EHR (4.3%; 18 of 422 results) and were rarely entered directly into the EHR. The frequency of manual entry of results may be due to the spectrum of tests examined in this study, which included surgical pathology and cytology reports (Table 4). Other tests that were less frequently electronically interfaced included HIV-1 quantitative polymerase chain reaction, serum protein electrophoresis, and factor V Leiden polymerase chain reaction.

Most—84.8% (128 of 151)—of the results reviewed that were received from reference laboratories were electronically transferred through the LIS to the EHR. Small numbers of reference laboratory results were manually entered into the LIS or scanned into the EHR (Table 5). As for in-house–performed services, anatomic pathology reports were frequently entered manually into the LIS or scanned into the EHR.

Most of the test results reviewed by participants contained all of the reporting elements considered necessary for a complete report (Table 6). There was little difference between the completeness of reporting elements between normal and abnormal test results. The name/address of the performing laboratory and date/time of test result/report were the elements most frequently missing on result review. More than 99% (1038 of 1045) of test results reviewed were appropriately formatted, whether the result was observed in the EHR on a computer screen or a paper printout of the EHR result (Table 7). There was no clear difference in the appropriateness of formatting between normal and abnormal test results. However, almost 9% (85 of 1027) of the test results reviewed on an EHR computer screen contained extraneous information and/or information that did not need to be reported. This extra information can potentially complicate test result interpretation by the HCP.

Participating institutions provided information regarding their IT practices related to laboratory testing (Table 8). There was a broad distribution concerning the percent of tests that were electronically ordered by the provider. All laboratories used an LIS, and all institutions had implemented an EHR. Approximately half of laboratories also used middleware. Slightly more than half (55.8%; 24 of 43) of the institutions had test information (eg, specimen type, patient preparation, turnaround time, etc) available in the electronic ordering system. Most (70.5%; 31 of 44) of the institutions reported that most test results transmitted to the EHR were routed to an “inbox” or other electronic system used by the ordering HCP. In addition, most (79.5%; 35 of 44) participants reported that most point-of-care test results were available in the EHR. Slightly more than half (54.5%; 24 of 44) of the participants stated that patients were able to review their laboratory results within a personal health record or other electronic patient portal.

Of the 45 institutions participating in the study, 41 (91%) were from the United States. The remaining 4 institutions were from Saudi Arabia (2), Brazil, and Canada. Within the last 2 years prior to data collection, 77.8% (35 of 45) of participating laboratories were inspected by the CAP. The size of most hospitals with hospital-based laboratories (69.8%; 30 of 43) was less than 300 occupied beds. Approximately one-third of participating laboratories were teaching hospitals that also trained pathology residents. Overall, 45.5% (20 of 44) of institutions were located in cities and 75.0% (33 of 44) were nongovernmental. Mean test volumes for participating laboratories were 1 809 998 for clinical pathology tests, 33 457 for surgical pathology tests, and 14 744 for gynecologic cytology.

This Q-Probes study focused on the electronic reporting of laboratory results within the EHR. It is important for laboratories to verify test result accuracy, completeness, and usability within the EHR because HCPs frequently manage test results and other patient information directly within these systems. There are also several IT requirements for reporting laboratory test results that must be met, as stipulated in the Code of Federal Regulations⁴  and the CAP's Laboratory Accreditation Program Laboratory General Checklist³  (Table 9). The Code of Federal Regulations requires that a manual or electronic system be in place to ensure test results are reliably sent from the point of data entry to the final report destination. This requirement applies to both results transmitted via an electronic interface and manually entered results, and includes results for tests performed in-house and at reference laboratories.

The IT requirements for test resulting can be broadly categorized into “accuracy,” “completeness,” and “usability” of test result information that is transmitted to the EHR. The participants of this Q-Probes study documented that 99.3% (1052 of 1059) of results reviewed were accurately transmitted to the EHR. This finding is expected in that most results are electronically, not manually, entered. However, there was lower compliance for completeness of test results, with only 69.6% (732 of 1051) of the test results containing all of the essential reporting elements outlined above. Institutions at which less than half of test orders are entered electronically had lower test result completeness rates. This may be due to less developed electronic interfaces at these institutions. The report components that were most commonly missing included the “name/address of performing lab” and the “date/time of test result/report.” The most important information (eg, result, reference range, interpretation) should be prominently displayed and not obscured by other elements that are less important to the HCP (eg, disclaimers, test methodology, etc). Reference intervals were available within the EHR for 97.1% (749 of 771) of the results reviewed. The required reporting elements also apply to point-of-care test results.

Most laboratories in this study electronically transmitted results from the performing instrument to the LIS and then to the EHR for commonly performed tests, including platelet counts, creatinine, estimated glomerular filtration rate, and international normalized ratio. Tests that are performed less frequently and/or may not be automated (eg, quantitative HIV polymerase chain reaction, protein electrophoresis, and factor V Leiden mutational assay) often required manual result entry. Electronic interfaces were used to transmit most (84.8%; 128 of 151) of the results reviewed for tests performed at the participant's primary reference laboratory. A small number of laboratories manually entered referred testing results into the LIS or scanned/copied results directly into the EHR. The CAP Laboratory Accreditation Program Checklist requirements (GEN.41440) stipulate that essential elements of referred test results be reported by the referring laboratory as received from the reference laboratory, without alternations that could affect clinical interpretation. Formatting of results can be altered when results are electronically transmitted from an LIS to an EHR. For example, tables, underlining, and alignment are often lost or transmitted inaccurately during electronic transfer. This can be especially problematic for surgical pathology and cytology reports that are often prepared using text editors with richer formatting features. In these cases, creating a report using an open standard for electronic document exchange (eg, PDF format) that can be directly viewed in the EHR without the need for discrete data transfer will preserve formatting. Maintaining consistency of formatting for test results may also improve their usability. Laboratory IT specialists will need to develop particularly rigorous strategies for formatting results from newer genomics technologies.⁵ 

Electronic interfaces have enabled more rapid and accurate transmission of laboratory results. However, it is critical that laboratories review transmitted results to ensure they are complete, accurate, and in a maximally usable format. Laboratory results should be reviewed before going live with a new interface that transmits results to the EHR, when changes are made at the laboratory or EHR level that could alter test resulting, and periodically. In this study, a relatively low percentage of laboratories were shown to verify the transmission of data to the EHR at least once a year or every other year. The CAP Laboratory General Checklist³  requires that the CAP laboratory director review and approve the content and format of paper and electronic patient reports at least every 2 years.

A strength of this study was that participants evaluated both clinical and anatomic pathology results because the content and format of these reports differ significantly. However, the study did have limitations, the most important being the variability of the persons who assessed the reports. Participants did not have difficulty determining whether a required reporting element was present or absent, but this may not be true for the more subjective assessments of report formatting and usability. Although guidelines were provided as described in “Materials and Methods” for judging these aspects, each participating laboratory assessed the quality of its own reports; significant variability between sites in the more subjective assessments cannot be excluded. Furthermore, the background and experience of evaluators could also influence how they critiqued reports. Participants did not specify who at their institution evaluated the reports (eg, pathologist, laboratory personnel, technologist, clinician, etc), and it is possible that more than one individual at a site was needed to review all tests. Finally, some participants did not fully complete data collection forms for all 16 tests. This frequently occurs in multisite Q-Probes studies when participants have difficulty finding requested information or do not fully understand the data collection instructions.

The methods outlined in this study can be used to help laboratories meet requirements for verifying laboratory test resulting within an EHR. The verification process should include careful scrutiny for the required reporting elements described in this study that are considered best practices and are required by regulatory agencies. In some medical systems, it may be necessary to review electronic results in more than one electronic system. For example, because laboratories are increasingly providing patients access to laboratory test information, the accuracy and usability of these results within patient portals should also be verified.^6,7  In fact, many medical laboratories in the United States are required to provide patient access to their laboratory test results. Although patients are generally satisfied when they can view their test results online, it remains unclear whether patient access to laboratory reports and other elements of their EHR improves the quality and safety of health care.⁸  Medical and laboratory professionals express concerns that patients may not fully appreciate the implications of their test results. This is particularly true for anatomic and cytology reports that are inherently difficult for most patients to understand. Finally, laboratories should also focus on the usability of test resulting to reduce the risk of the HCP misinterpreting a test result. This may require interaction with a medical informatics officer or other clinicians who participate in institutional IT design and implementation. Overall, laboratory professionals should be actively involved in the development and maintenance of electronic test ordering⁹  and resulting systems¹⁰  in collaboration with their IT specialists.