Pathology Trainees Gain Clinical Pathology Experience as Lab Consultants Through Auditing Myeloid Mutation Panel Send-Out Tests: Hitting Two Birds With One Stone?

A resident-driven intervention was implemented at the University of Maryland Medical Center in Baltimore. During a 6-month period, from July 27, 2020, to January 29, 2021, senior pathology residents rotating on clinical chemistry reviewed all MMP send-out requests, which included chart review in real time to evaluate indications for testing and correlation with preliminary bone marrow biopsy results. Appropriate test indications included acute myeloid leukemia, non-M3, myelodysplastic syndromes, BCR-ABL⁻ myeloproliferative neoplasms, and myelodysplastic syndrome/myeloproliferative neoplasm overlap disorders as outlined by the World Health Organization,³  primary literature, and the test manufacturer. A test was considered inappropriate if it did not meet one of these indications, was a duplicate order, or was ordered <2 months from a prior MMP. In order to study the effect of the resident-driven audit, a retrospective review of the myeloid malignancy send-out requests from the 6 months prior (January 6, 2020, to July 26, 2020) was also performed. The same criteria were used during both the retrospective review and the audit period.

During the intervention period, we introduced practice alterations with explicit participation by a multidisciplinary team including a pathology senior resident, an attending hematopathologist, and clinical laboratory send-out staff. These included new guidelines and a distinct process for evaluation of MMP appropriateness (Figure 1).

At least twice daily, residents would review the myeloid mutation test requests. For each request, the electronic health record and Cerner pathology database were investigated for the following: patient's suspected diagnosis, prior bone marrow aspirate, and findings, as well as any recent MMP orders. The starting point of the algorithm was whether or not a patient had a myeloid disease. All testing that was not performed for myeloid disease was further investigated, either through direct contact with the ordering provider by the resident if an obvious incorrect indication was present, or after consultation with a hematopathologist in cases where an indication was uncertain from chart review. Cases brought to the attention of the ordering provider that still did not meet indication upon further clarification were permitted on a case-by-case basis depending on the ordering provider's rationale for testing and persistence.

In cases where testing was determined to be inappropriate by either the attending physician or resident, discussion with the ordering provider would occur on the same day. The semistructured conversation would focus on the ordering provider's reasoning for testing appropriateness and often led to resident feedback on preliminary bone marrow findings, determination if these results changed the ordering provider's need for the test, and education regarding appropriate indications for testing to the provider.

Information regarding the status of tests was then sent by the resident to the laboratory send-out staff via email by the end of each workday.

Information pertinent to the MMP requests was collected in a password-protected Excel file (Microsoft, Redmond, Washington). Pertinent laboratory data recorded for each request included the order date, patient name, and accession number for review in the electronic medical record, appropriateness of test order, and whether or not the test was permitted. This Excel file was managed by a senior resident rotating on the clinical chemistry lab director service and sent to the next resident on the service at the conclusion of each rotation. Differences between the number of inappropriate tests as well as the proportion of inappropriately permitted tests before and after intervention were determined using χ² and Fisher exact tests as appropriate using Rstudio (version 1.2.5042). Data visualization was performed using the R statistical programming language and tidyr and ggplot2 packages.^4,5 

During the 6-month study period, the total number of MMP tests ordered was 234. Of the 234 tests, most (78.2%; n = 183) were considered appropriate orders. Of the 51 tests that were considered inappropriate orders, discussion with the clinical team occurred in all circumstances, and 76% were successfully canceled (n = 39), with occasional inappropriate tests permitted on a case-by-case basis (n = 12; Table). Reasons for inappropriate orders were predominantly an inappropriate indication for test selection (84.3%; n = 43), but also duplicate or repeat orders (15.7%; n = 8). Inappropriate indications varied but included a diagnosis of nonmyeloid hematologic malignancy. Although there was some variability during the course of the study period, in general, rates of inappropriate testing, whether due to inappropriate indication or duplicate order, did not change (Figure 2). Overall, this intervention led to a $62 400 cost savings during the 6-month period, with a potential savings of $81 600 if none of the inappropriate tests were permitted. Despite efforts to educate on a case-by-case basis, the percentage of indicated tests during the course of the intervention showed only a slight positive trend in more appropriate test use.

In the 6-month period prior to the study, 195 MMP tests were ordered, with 154 considered appropriate (79.0%; n = 41). Of the 41 inappropriate tests, 5 were duplicate orders (12.2%; n = 5). All inappropriate tests were permitted during this period. There was no significant difference seen in the proportion of inappropriate tests between the control and intervention periods (control, 21.0%; intervention, 21.8%; P = .90, χ² test); however, there was a significant increase in canceled tests during the intervention period (control, 0%; intervention, 76.5%; P < .001, Fisher exact test; Figure 2).

Although somewhat controversial in the primary literature, an inappropriate laboratory test is generally defined as “any test order that is in violation of a guideline produced by a government or professional society,”⁶  and it is currently estimated that 20% of lab testing is inappropriate by this definition.^7–9  As recommended by the World Health Organization and in the primary literature, MMP testing should be pursued only in the context of a myeloid disease process.³  At our institution, MMP testing was readily recognized by hematopathologists and lab clinicians alike as a send-out test that would benefit from resident intervention for several reasons. The first was that it is an overused laboratory test with high-volume ordering, which had both cost implications and potential implications for patient care. Inappropriate laboratory testing has been described in the literature as posing a threat to clinical care because it can lead to unnecessary phlebotomy or diagnostic error that can contribute to subsequent patient harm.^6–8,10 

Following the testing appropriateness algorithm, residents were able to execute a systematic, consistent, and targeted approach to test evaluation during a 6-month period and canceled a significant portion of inappropriate tests. The most frequent cause of inappropriate ordering was an incorrect indication selected, which was often due to ordering a test before firmly establishing a diagnosis, or a provider's unfamiliarity with the test indications. The current literature on inappropriate test ordering shows similar reasons for inappropriate ordering, including ordering provider lack of awareness and knowledge of testing as well as personal factors, such as feelings of insecurity.⁷  Additionally, because of the layout of the electronic medical record ordering system, tests are often ordered prior to establishing a pathologic diagnosis. During the course of the intervention, the frequency of inappropriate orders did not vary substantially from month to month, likely because of the fact that there were several different providers involved in the inappropriate ordering and it would take a longer duration of time to see the long-term impact of the intervention. Review of the literature shows evidence to support that auditing a clinician's lab decisions and offering feedback can result in positive change,¹¹  although other reports show a large range at which ordering practices change following feedback and that the long-term impact of these types of interventions is lacking.^7,10 

Despite not seeing an immediate, significant positive change in physician ordering practices during the course of the intervention, the intervention did have several other positive impacts. One important result of the intervention was increased resident awareness and familiarity with aspects of lab use and management, which are areas of CP that are fundamental to improving the CP training experience¹² ; notably, these areas are also often cited by the newly trained pathologist as areas with the lowest preparedness.¹³  Similar residency initiatives in lab use brought up concerns regarding the intensive, manual basis for review that could take residents away from other key clinical service activities; ultimately, the conclusion from these studies was that there was educational value in terms of both medical knowledge and laboratory management domains through these initiatives.^9,14 

Another result of the intervention was more experience working in an interprofessional team through consultation of both the hematopathology attending as well as ordering providers from various services. Prior studies have described the possible adversarial role that the resident has to play in communicating lab testing decisions, which is often a difficult conversation to have with an attending-level provider as a resident.¹⁵  However, this sort of communication and experience is integral to pathology training, and is in fact a key component of the Accreditation Council for Graduate Medical Education pathology milestones.^16,17  One way to mitigate the potential for conflict is to ensure that the key ordering providers agree with the algorithm upon which the interventions are based. In the case of the MMPs, these key stakeholders are the oncologists who care for patients with myeloid malignancies. Guidelines for testing intervention therefore have been developed in collaboration with the oncologists at our institution.

Having pathology residents serving as the bridge between the laboratory and the clinician has a 2-fold effect: (1) the lab is no longer viewed as a commodity for lab testing, but as a source of interpretation and physician guidance and (2) a sense of teamwork is provided because the lab and clinical providers become directly involved in the patient's care. These points have been well described in lab use literature, stressing the need for cooperation and a clinical-laboratory interface, which also leads to better physician knowledge and awareness of testing.¹⁸  These are key aspects of modern laboratory medicine, whereby laboratory professionals contribute to the interpretation of laboratory testing and have a direct clinical and economic impact.^18–21 

Finally, there is the potential for significant cost savings from this type of intervention. Other resident-led initiatives have considered the “time-cost” of manual intervention when interpreting the cost savings of these types of intervention—meaning the cost of time of the resident, laboratory, hematopathology attending, and ordering provider involved.⁹  Assessing the cost of that time is difficult given the variability in the time spent on each case and regional variations in salary. However, studies have concluded that the cost of test review is likely less than the cost of the send-out testing,⁹  which may be especially true of high-cost send-out testing like MMP orders, which are priced at approximately $1600 per test. Resident intervention during the 6-month period resulted in a cost savings of $62 400. These savings can then have potential implications on the hospital's annual reference laboratory budget, for instance, freeing up money for additional testing to meet their patients' and clinicians' needs.

There are obvious limitations to our study. First, this was a small, single-center investigation that was only 6 months in length, so understanding whether or not the intervention improved ordering practices was not established; however, other studies have shown that similar resident initiatives can lead to better ordering practices over the long term.^22,23  We did not investigate the credentials of the ordering provider and the effect that may have had on ordering practices during the course of our intervention; prior studies have shown that the correctness of lab test use does not improve with increased experience, and thus we suspect that the provider's role would have little impact on ordering practices.²⁴  Finally, there was some variability in resident ability to assess appropriateness of testing with greater hematopathologist consultation and time spent on each case, limiting the overall cost savings and educational value when considering the time spent evaluating each case. The expectation, however, is that as residents gain experience and familiarity with the workflow and test appropriateness algorithm, the time-consuming nature of the task would decrease. Limitations described here, such as variability of the effectiveness of individuals conducting the manual review and scalability of the intervention, may also point to decreased applicability of this intervention to other institutions.¹⁵ 

Involving the pathology resident as part of the laboratory professional team serves to not only enhance lab use management but to facilitate core CP training. This beneficial aspect to training can somewhat offset the manual intensity and time-consuming nature of audits, and can lead to decreases in inappropriate test use, with potentially significant clinical and economic impacts. Integration of pathology residents into these efforts can not only help bridge the lab and the ordering clinical providers but can also help shift the view of the lab as merely a commodity for testing to the modern laboratory medicine paradigm, whereby laboratory professionals serve as consultants to ordering providers. To our knowledge, we are the first to report results of a pathology resident–led intervention targeting inappropriate send-out testing, and we describe a teamwork-based approach that contributes to cost savings, resident education, and the lab-clinician interface.