Procedural experience for residents and fellows is critical for achieving competence, and documentation of procedures performed is required. Procedure logs serve as the record of this experience, but are commonly generated manually, require substantial administrative effort, and cannot be corroborated for accuracy.
We developed and implemented a structured clinical-educational report template (CERT), which automatically generates procedure logs directly from the clinical record.
Our CERT aimed to replace the post-procedure note template for our cardiac catheterization laboratory and was incorporated into the electronic health record system. Numbers of documented procedures in automated CERT-derived procedure logs over a 1-year post-intervention period (2018–2019) were compared to manual logs and corrected for clinical volume changes. The CERT's impact on fellowship experience was also assessed.
Automated CERT procedure logs increased weekly procedural documentation over manual procedure logs for total procedures (24.2 ± 6.1 vs 17.1 ± 6.8, P = .007), left heart catheterizations (14.5 ± 3.6 vs 10.8 ± 4.2, P = .039), total procedural elements (40.2 ± 11.4 versus 20.9 ± 12.5, P < .001), and captured procedural details not previously documented. The CERT also reduced self-reported administrative time and improved fellowship experience.
A novel CERT allows procedure logs to be automatically derived from the clinical record and increased the number of documented procedures, compared to manual logging. This innovation ensures an accurate record of procedural experience and reduces self-reported non-educational administrative time for cardiology fellows.
To develop a structured clinical-educational report template (CERT), which generates procedure logs automatically from the clinical record.
A novel CERT increases the number of documented procedures and reduces self-reported administrative time for cardiology fellows, compared to manual logging.
This CERT is focused on interventional cardiology procedures, although is adaptable for any procedural specialty.
A novel CERT allows procedure logs to be automatically derived from the clinical record, ensures an accurate record of procedural experience, and improves cardiology fellows' perception of fellowship experience.
Gaining proficiency in procedures is a core component of medical training and lays the foundation for high-quality independent practice. Invasive specialties have specific procedural volume requirements to achieve competency, and training programs are required to have procedural logging systems to document this experience. These systems commonly require manual data entry by the trainees themselves. For example, surgical specialties use the Accreditation Council for Graduate Medical Education (ACGME) case log system, which requires trainees to manually enter procedures they perform. Recent investigations of these ACGME case logs among general surgery residents have suggested that they are frequently inaccurate and markedly underreport actual operative experience.1–3 In medical procedural specialties such as cardiology, the accuracy of procedure logging systems has not been studied.
Structured reporting is a process in which clinical data entry is codified so that it can be used for automated reporting. Structured reporting has been endorsed by the American College of Cardiology as a means to track clinical performance and quality metrics, and shows promise as a means to document procedural experience for medical trainees.4
We hypothesized that a structured clinical-educational report template (CERT) with automated procedure log generation directly from the electronic health record (EHR) would improve the accuracy of procedural documentation for cardiology fellows. Additionally, we sought to assess the impact of an automated procedure logging system on non-educational administrative time and fellowship experience.
Setting and Participants
The subject of this investigation was the procedural documentation of first- and second-year cardiology fellows during rotations in the cardiac catheterization laboratory at the University of Chicago Medicine (UCM) from July 1, 2017 to November 30, 2019.
A CERT was developed to replace standard post-procedure note templates used by cardiology fellows after all cardiac catheterization procedures at UCM within the institutional EHR (Epic Systems Corp, Verona, WI). The CERT was implemented on November 10, 2018, and CERT-derived automated procedure logs were obtained for each fellow 1-year post-intervention (November 30, 2019).
CERT Development and Implementation
The CERT consisted of redesigned fields from the existing post-procedure note template to capture data in a structured format. A second-year cardiology fellow (E.C.A.), who previously completed the Epic Physician Build Basic course (free 3-day course), designed the CERT and served as the content expert. Select fields were replaced with SmartLists, which are pre-populated lists of frequently used entries that reduce documentation time for the user. The SmartLists were linked to SmartData Elements to store the selections discretely on note signing. Building the template with the guidance of the hospital Epic support team took approximately 10 hours. The CERT was tested in a preproduction Epic environment to ensure accurate data capture. The CERT added an average of 20 seconds to the time needed for post-procedure note completion.
The Epic team performed validation and workflow testing standard with all new Epic implementations over a 2-month period after CERT implementation. For 200 procedures, the CERT captured the entered data, including the involved fellow, with 100% accuracy.
One year after implementation, a cardiac catheterization procedures report was generated by the Epic team using Epic Clarity for all procedures since implementation. The report included all SmartData Elements for each procedure grouped for each individual physician and was used for this analysis.
Subsequently, automatic monthly reports in Microsoft Excel are generated and forwarded to fellowship program leadership. Individual monthly logs are then distributed to each trainee. Each year, the “Performing Physicians” SmartList (available as online supplementary data) is updated by our Epic team to include new fellows and remove graduates. This update takes approximately 5 minutes.
Outcomes and Analysis
Manual procedure logs in the digital platform MedHub, from July 1, 2017 to November 10, 2018, were compared to CERT-derived procedure logs in the year following implementation. Procedures for each fellow were divided by the number of weeks spent on cardiac catheterization rotation, to establish weekly procedures logged.
To correct for the potential impact of changing cardiac catheterization laboratory volume between 2017 and 2019, before and after CERT implementation, procedures were adjusted based on total UCM cardiac catheterization laboratory procedures to normalize to 2019 volume. UCM cardiac catheterization laboratory volume in 2019 increased 4.6% over 2018 volume and 15.1% over 2017 volume. Thus, procedure totals logged for each fellow in 2017 were multiplied by 1.151, and the total number of procedures logged in 2018 was multiplied by 1.046.
Independent sample t tests were performed to compare weekly procedures logged between subspecialties and fellows using manual logging versus CERT logging systems.
To assess the perceived impact of the automated CERT-based procedure logging system on trainee non-educational administrative time and fellowship experience, all fellows completed an anonymous survey at the conclusion of the study period (available as online supplementary data). The survey was created by the study team and distributed electronically. Electronic reminders were sent to increase participation. The UCM Institutional Review Board approved this study.
A total of 11 fellows, accounting for 21 individual fellowship years, participated in both the manual and CERT procedure logging systems between July 1, 2017 and November 30, 2019. More than 1400 procedures were logged manually in MedHub between July 1, 2017 and CERT implementation on November 10, 2018. Between this launch date and November 30, 2019, the CERT system was used 1725 times.
The per-week procedure volumes, corrected for changes in overall catheterization laboratory volume over time, are shown in Figure 1. CERT-derived procedure logs captured a significantly higher rate of total procedures (24.2 ± 6.1 versus 17.1 ± 6.8, P = .007), and left heart catheterizations (14.5 ± 3.6 versus 10.8 ± 4.0, P = .039), compared to manual logs. Left heart catheterization is reported separately as it the most common individual procedure performed. Additionally, the CERT system captured twice as many discrete procedural events/elements as the manual procedure logs (40.2 ± 11.4 vs 20.9 ± 12.5, P < .001). These elements included important procedural skills embedded in a single procedure. For example, radial artery access or coronary physiology assessment may occur within a left heart catheterization.
A comparison between manual and automated CERT-derived procedure logs among all procedures and procedure elements is shown in Figure 2. Automated procedure logging was associated with a higher rate of reporting for each procedure and all procedural elements.
All 11 UCM cardiology fellows included in this study completed the survey at the end of the study period. Regarding manual procedure logging, only 36% (4 of 11) reported that they felt completing accurate procedure logs was “very important,” 45% (5) as “somewhat important, and 18% (2) as “not important.” When asked to estimate the hours they spent manually logging for each catheterization rotation week, 55% (6 of 11) reported 1 to 3 hours and 27% (3) reported 3 to 5 hours. When asked what impact automated procedure logging had on their fellowship experience, 73% (8 of 11) reported a “very positive impact” and 27% (3) a “somewhat positive impact.” No one reported “no impact.”
Compared to manual procedure logging, CERT-derived procedure logs led to a significantly higher rate of procedure reporting with increased detail captured per case. This innovation also reduced self-reported non-educational administrative time for trainees and resulted in improved perceptions of fellowship experience.
Procedural experience is a core portion of training in a variety of medical specialties. While procedural volume itself does not infer competency, it serves as an important surrogate, as there is an inverse correlation between operator procedural volume and peri-procedural complications.5,6 Procedure logs serve as the official educational record of procedural volume for trainees, thus their accuracy is critical for assessments of procedural competence. We find that our CERT increases the number of procedures logged and enhances the detail of the procedural record.
Multiple prior investigations have reported systematic inaccuracies and underreporting of actual operative experience.1–3 For example, Naik et al found that ACGME case logs submitted by surgical interns missed 28% of actual cases performed when compared to an institutional automated operative reporting system.3 Less is known about the accuracy of procedural logging systems for medical specialties. Our study is the first to investigate an automated solution for procedure logs among cardiovascular disease trainees.
The elimination of the administrative time required for manual logging is of critical importance to medical trainees. In addition to adding duty hours, time spent performing nonclinical work has been implicated as a factor in physician burnout.7,8 Medical trainees are particularly vulnerable as they complete this additional documentation during what is now being referred to as “pajama time.”9 It is paramount that training programs employ time-efficient technological approaches to protect trainees' well-being.10,11
Additionally, CERT minimizes the risk of exposure of personal health information that can occur in external platforms. As all data are housed within the EHR, a CERT benefits from the full breadth of institutional resources for data security, backup, and analysis.
Limitations of the CERT approach include the technical burden of EHR implementation, because it required support from our Epic team. Our intervention also focused on interventional cardiology procedures, though it could be easily adapted for any procedural specialty. While we were unable to compare logging methods concurrently, careful corrections for clinical volume changes were made. Finally, the fact that our CERT was developed in an Epic EHR system and compared to manual logging in MedHub may limit generalizability. However, given the prevalence of structured reporting capabilities, we anticipate our CERT would be readily adaptable for a majority of EHR vendors. In this spirit, we have shared the full structure of CERT (available as online supplementary data), so that others may implement it in their training programs and further innovate as needed.
We found that a CERT improves accuracy, completeness, and detail of procedure logs, and reduces self-reported non-educational administrative time for cardiology trainees.
Editor's Note: The online version of this article contains the survey of fellows who participated in both manual and automated procedure log systems, example of CERT note completion in the Epic electronic health record, and full hierarchal structure of CERT structured reporting fields.
Funding: The authors report no external funding source for this study.
Conflict of interest: The authors declare they have no competing interests.