ABSTRACT

Background

Procedural experience for residents and fellows is critical for achieving competence, and documentation of procedures performed is required. Procedure logs serve as the record of this experience, but are commonly generated manually, require substantial administrative effort, and cannot be corroborated for accuracy.

Objective

We developed and implemented a structured clinical-educational report template (CERT), which automatically generates procedure logs directly from the clinical record.

Methods

Our CERT aimed to replace the post-procedure note template for our cardiac catheterization laboratory and was incorporated into the electronic health record system. Numbers of documented procedures in automated CERT-derived procedure logs over a 1-year post-intervention period (2018–2019) were compared to manual logs and corrected for clinical volume changes. The CERT's impact on fellowship experience was also assessed.

Results

Automated CERT procedure logs increased weekly procedural documentation over manual procedure logs for total procedures (24.2 ± 6.1 vs 17.1 ± 6.8, P = .007), left heart catheterizations (14.5 ± 3.6 vs 10.8 ± 4.2, P = .039), total procedural elements (40.2 ± 11.4 versus 20.9 ± 12.5, P < .001), and captured procedural details not previously documented. The CERT also reduced self-reported administrative time and improved fellowship experience.

Conclusions

A novel CERT allows procedure logs to be automatically derived from the clinical record and increased the number of documented procedures, compared to manual logging. This innovation ensures an accurate record of procedural experience and reduces self-reported non-educational administrative time for cardiology fellows.

Objectives

To develop a structured clinical-educational report template (CERT), which generates procedure logs automatically from the clinical record.

Findings

A novel CERT increases the number of documented procedures and reduces self-reported administrative time for cardiology fellows, compared to manual logging.

Limitations

This CERT is focused on interventional cardiology procedures, although is adaptable for any procedural specialty.

Bottom line

A novel CERT allows procedure logs to be automatically derived from the clinical record, ensures an accurate record of procedural experience, and improves cardiology fellows' perception of fellowship experience.

Introduction

Gaining proficiency in procedures is a core component of medical training and lays the foundation for high-quality independent practice. Invasive specialties have specific procedural volume requirements to achieve competency, and training programs are required to have procedural logging systems to document this experience. These systems commonly require manual data entry by the trainees themselves. For example, surgical specialties use the Accreditation Council for Graduate Medical Education (ACGME) case log system, which requires trainees to manually enter procedures they perform. Recent investigations of these ACGME case logs among general surgery residents have suggested that they are frequently inaccurate and markedly underreport actual operative experience.13  In medical procedural specialties such as cardiology, the accuracy of procedure logging systems has not been studied.

Structured reporting is a process in which clinical data entry is codified so that it can be used for automated reporting. Structured reporting has been endorsed by the American College of Cardiology as a means to track clinical performance and quality metrics, and shows promise as a means to document procedural experience for medical trainees.4 

We hypothesized that a structured clinical-educational report template (CERT) with automated procedure log generation directly from the electronic health record (EHR) would improve the accuracy of procedural documentation for cardiology fellows. Additionally, we sought to assess the impact of an automated procedure logging system on non-educational administrative time and fellowship experience.

Methods

Setting and Participants

The subject of this investigation was the procedural documentation of first- and second-year cardiology fellows during rotations in the cardiac catheterization laboratory at the University of Chicago Medicine (UCM) from July 1, 2017 to November 30, 2019.

Intervention

A CERT was developed to replace standard post-procedure note templates used by cardiology fellows after all cardiac catheterization procedures at UCM within the institutional EHR (Epic Systems Corp, Verona, WI). The CERT was implemented on November 10, 2018, and CERT-derived automated procedure logs were obtained for each fellow 1-year post-intervention (November 30, 2019).

CERT Development and Implementation

The CERT consisted of redesigned fields from the existing post-procedure note template to capture data in a structured format. A second-year cardiology fellow (E.C.A.), who previously completed the Epic Physician Build Basic course (free 3-day course), designed the CERT and served as the content expert. Select fields were replaced with SmartLists, which are pre-populated lists of frequently used entries that reduce documentation time for the user. The SmartLists were linked to SmartData Elements to store the selections discretely on note signing. Building the template with the guidance of the hospital Epic support team took approximately 10 hours. The CERT was tested in a preproduction Epic environment to ensure accurate data capture. The CERT added an average of 20 seconds to the time needed for post-procedure note completion.

The Epic team performed validation and workflow testing standard with all new Epic implementations over a 2-month period after CERT implementation. For 200 procedures, the CERT captured the entered data, including the involved fellow, with 100% accuracy.

One year after implementation, a cardiac catheterization procedures report was generated by the Epic team using Epic Clarity for all procedures since implementation. The report included all SmartData Elements for each procedure grouped for each individual physician and was used for this analysis.

Subsequently, automatic monthly reports in Microsoft Excel are generated and forwarded to fellowship program leadership. Individual monthly logs are then distributed to each trainee. Each year, the “Performing Physicians” SmartList (available as online supplementary data) is updated by our Epic team to include new fellows and remove graduates. This update takes approximately 5 minutes.

Outcomes and Analysis

Manual procedure logs in the digital platform MedHub, from July 1, 2017 to November 10, 2018, were compared to CERT-derived procedure logs in the year following implementation. Procedures for each fellow were divided by the number of weeks spent on cardiac catheterization rotation, to establish weekly procedures logged.

To correct for the potential impact of changing cardiac catheterization laboratory volume between 2017 and 2019, before and after CERT implementation, procedures were adjusted based on total UCM cardiac catheterization laboratory procedures to normalize to 2019 volume. UCM cardiac catheterization laboratory volume in 2019 increased 4.6% over 2018 volume and 15.1% over 2017 volume. Thus, procedure totals logged for each fellow in 2017 were multiplied by 1.151, and the total number of procedures logged in 2018 was multiplied by 1.046.

Independent sample t tests were performed to compare weekly procedures logged between subspecialties and fellows using manual logging versus CERT logging systems.

To assess the perceived impact of the automated CERT-based procedure logging system on trainee non-educational administrative time and fellowship experience, all fellows completed an anonymous survey at the conclusion of the study period (available as online supplementary data). The survey was created by the study team and distributed electronically. Electronic reminders were sent to increase participation. The UCM Institutional Review Board approved this study.

Results

A total of 11 fellows, accounting for 21 individual fellowship years, participated in both the manual and CERT procedure logging systems between July 1, 2017 and November 30, 2019. More than 1400 procedures were logged manually in MedHub between July 1, 2017 and CERT implementation on November 10, 2018. Between this launch date and November 30, 2019, the CERT system was used 1725 times.

The per-week procedure volumes, corrected for changes in overall catheterization laboratory volume over time, are shown in Figure 1. CERT-derived procedure logs captured a significantly higher rate of total procedures (24.2 ± 6.1 versus 17.1 ± 6.8, P = .007), and left heart catheterizations (14.5 ± 3.6 versus 10.8 ± 4.0, P = .039), compared to manual logs. Left heart catheterization is reported separately as it the most common individual procedure performed. Additionally, the CERT system captured twice as many discrete procedural events/elements as the manual procedure logs (40.2 ± 11.4 vs 20.9 ± 12.5, P < .001). These elements included important procedural skills embedded in a single procedure. For example, radial artery access or coronary physiology assessment may occur within a left heart catheterization.

Figure 1

Corrected Procedure Volume Reported in Manual vs Automated CERT-Derived Procedure Logs

Note: Total Logged Events–all elements captured and recorded in procedure logs, even if more than one occurred in the same procedure (eg, a radial artery access for a coronary angiogram would be considered 2 events, radial access and coronary angiogram); Total Logged Procedures–all individual discrete procedures captured (eg, left heart catheterization, right heart catheterization, pericardiocentesis, etc); NB–the “capture” of specific procedure and procedural elements by manual vs automated CERT-derived procedure logs are displayed in Figure 2.

Figure 1

Corrected Procedure Volume Reported in Manual vs Automated CERT-Derived Procedure Logs

Note: Total Logged Events–all elements captured and recorded in procedure logs, even if more than one occurred in the same procedure (eg, a radial artery access for a coronary angiogram would be considered 2 events, radial access and coronary angiogram); Total Logged Procedures–all individual discrete procedures captured (eg, left heart catheterization, right heart catheterization, pericardiocentesis, etc); NB–the “capture” of specific procedure and procedural elements by manual vs automated CERT-derived procedure logs are displayed in Figure 2.

Figure 2

Manual vs Automated CERT-Derived Individual Procedure and Procedural Element Capture

Note: Automated procedures and procedural elements listed were those derived directly from the CERT procedure logs. Manual procedure and procedural elements were those recorded in MedHub, which is prepopulated with a list of common catheterization laboratory procedures, including left and right heart catheterization, percutaneous coronary intervention, mechanical circulatory support, pericardiocentesis, temporary venous pacing, and coronary imaging/physiology studies. There is also an “other” category allowing free-text entry to report any procedure or procedural element that does not fall into the above predefined categories (including but not limited to STEMI, radial access) at the fellow's discretion.

Figure 2

Manual vs Automated CERT-Derived Individual Procedure and Procedural Element Capture

Note: Automated procedures and procedural elements listed were those derived directly from the CERT procedure logs. Manual procedure and procedural elements were those recorded in MedHub, which is prepopulated with a list of common catheterization laboratory procedures, including left and right heart catheterization, percutaneous coronary intervention, mechanical circulatory support, pericardiocentesis, temporary venous pacing, and coronary imaging/physiology studies. There is also an “other” category allowing free-text entry to report any procedure or procedural element that does not fall into the above predefined categories (including but not limited to STEMI, radial access) at the fellow's discretion.

A comparison between manual and automated CERT-derived procedure logs among all procedures and procedure elements is shown in Figure 2. Automated procedure logging was associated with a higher rate of reporting for each procedure and all procedural elements.

All 11 UCM cardiology fellows included in this study completed the survey at the end of the study period. Regarding manual procedure logging, only 36% (4 of 11) reported that they felt completing accurate procedure logs was “very important,” 45% (5) as “somewhat important, and 18% (2) as “not important.” When asked to estimate the hours they spent manually logging for each catheterization rotation week, 55% (6 of 11) reported 1 to 3 hours and 27% (3) reported 3 to 5 hours. When asked what impact automated procedure logging had on their fellowship experience, 73% (8 of 11) reported a “very positive impact” and 27% (3) a “somewhat positive impact.” No one reported “no impact.”

Discussion

Compared to manual procedure logging, CERT-derived procedure logs led to a significantly higher rate of procedure reporting with increased detail captured per case. This innovation also reduced self-reported non-educational administrative time for trainees and resulted in improved perceptions of fellowship experience.

Procedural experience is a core portion of training in a variety of medical specialties. While procedural volume itself does not infer competency, it serves as an important surrogate, as there is an inverse correlation between operator procedural volume and peri-procedural complications.5,6  Procedure logs serve as the official educational record of procedural volume for trainees, thus their accuracy is critical for assessments of procedural competence. We find that our CERT increases the number of procedures logged and enhances the detail of the procedural record.

Multiple prior investigations have reported systematic inaccuracies and underreporting of actual operative experience.13  For example, Naik et al found that ACGME case logs submitted by surgical interns missed 28% of actual cases performed when compared to an institutional automated operative reporting system.3  Less is known about the accuracy of procedural logging systems for medical specialties. Our study is the first to investigate an automated solution for procedure logs among cardiovascular disease trainees.

The elimination of the administrative time required for manual logging is of critical importance to medical trainees. In addition to adding duty hours, time spent performing nonclinical work has been implicated as a factor in physician burnout.7,8  Medical trainees are particularly vulnerable as they complete this additional documentation during what is now being referred to as “pajama time.”9  It is paramount that training programs employ time-efficient technological approaches to protect trainees' well-being.10,11 

Additionally, CERT minimizes the risk of exposure of personal health information that can occur in external platforms. As all data are housed within the EHR, a CERT benefits from the full breadth of institutional resources for data security, backup, and analysis.

Limitations of the CERT approach include the technical burden of EHR implementation, because it required support from our Epic team. Our intervention also focused on interventional cardiology procedures, though it could be easily adapted for any procedural specialty. While we were unable to compare logging methods concurrently, careful corrections for clinical volume changes were made. Finally, the fact that our CERT was developed in an Epic EHR system and compared to manual logging in MedHub may limit generalizability. However, given the prevalence of structured reporting capabilities, we anticipate our CERT would be readily adaptable for a majority of EHR vendors. In this spirit, we have shared the full structure of CERT (available as online supplementary data), so that others may implement it in their training programs and further innovate as needed.

Conclusions

We found that a CERT improves accuracy, completeness, and detail of procedure logs, and reduces self-reported non-educational administrative time for cardiology trainees.

References

References
1. 
Nygaard
RM,
Daly
SR,
Van Camp
JM.
General surgery resident case logs: do they accurately reflect resident experience?
J Surg Educ
.
2015
;
72
(6)
:
e178
e183
.
2. 
Ball
F,
Garwe
T,
Motghare
P,
Stamile
T,
Kim
J,
Mahnjen
H,
et al.
Evaluating coding accuracy in general surgery residents' Accreditation Council for Graduate Medical Education procedural case logs
.
J Surg Educ
.
2016
;
73
(6)
:
e59
e63
.
3. 
Naik
ND,
Abbott
EF,
Aho
JM,
Pandian
TK,
Thiels
CA,
Heller
SF,
et al.
The ACGME case log system may not accurately represent operative experience among general surgery interns
.
J Surg Educ
.
2017
;
74
(6)
:
e106
e110
.
4. 
Sanborn
TA,
Tcheng
JE,
Anderson
HV,
Chambers
CE,
Cheatham
SL,
DeCaro
MV,
et al.
ACC/AHA/SCAI 2014 health policy statement on structured reporting for the cardiac catheterization laboratory: a report of the American College of Cardiology Clinical Quality Committee
.
Circulation
.
2014
;
129
(24)
:
2578
609
.
5. 
Jolly
SS,
Cairns
J,
Yusuf
S,
Niemela
K,
Steg
PG,
Worthley
M,
et al.
Procedural volume and outcomes with radial or femoral access for coronary angiography and intervention
.
J Am Coll Cardiol
.
2014
;
63
(10)
:
954
963
.
6. 
McGrath
PD,
Wennberg
DE,
Dickens
JD,
Siewers
AE,
Lucas
FL,
Malenka
DJ,
et al.
Relation between operator and hospital volume and outcomes following percutaneous coronary interventions in the era of the coronary stent
.
JAMA
.
2000
;
284
(24)
:
3139
3144
.
7. 
Gardner
RL,
Cooper
E,
Haskell
J,
Harris
DA,
Poplau
S,
Kroth
PJ,
et al.
Physician stress and burnout: the impact of health information technology
.
J Am Med Inform Assoc
.
2019
;
26
(2)
:
106
114
.
8. 
Shanafelt
TD,
Dyrbye
LN,
Sinsky
C,
Hasan
O,
Satele
D,
Sloan
J,
et al.
Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction
.
Mayo Clin Proc
.
2016
;
91
(7)
:
836
848
.
9. 
Saag
HS,
Shah
K,
Jones
SA,
Testa
PA,
Horwitz
LI.
Pajama time: working after work in the electronic health record
.
J Gen Intern Med
.
2019
;
34
(9)
:
1695
1696
.
10. 
Seufert
TS,
Mitchell
PM,
Wilcox
AR,
Rubin-Smith
JE,
White
LF,
McCabe
KK,
et al.
An automated procedure logging system improves resident documentation compliance
.
Acad Emerg Med
.
2011
;
18
(Suppl 2)
:
54
58
.
11. 
Simpao
A,
Heitz
JW,
McNulty
SE,
Chekemian
B,
Brenn
BR,
Epstein
RH.
The design and implementation of an automated system for logging clinical experiences using an anesthesia information management system
.
Anesth Analg
.
2011
;
112
(2)
:
422
429
.

Author notes

Editor's Note: The online version of this article contains the survey of fellows who participated in both manual and automated procedure log systems, example of CERT note completion in the Epic electronic health record, and full hierarchal structure of CERT structured reporting fields.

Emeka C. Anyanwu, MSc, MD, is a Fellow in Cardiovascular Disease, University of Chicago Medicine; Victor Mor-Avi, PhD, is Research Professor, Director of Cardiovascular Research Training, University of Chicago Medicine; and R. Parker Ward, MD, is Director, Cardiovascular Disease Training Program, and Professor of Medicine, University of Chicago Medicine.

Funding: The authors report no external funding source for this study.

Competing Interests

Conflict of interest: The authors declare they have no competing interests.

Supplementary data