Abstract

Background

The electronic health record (EHR) includes a rich data set that may offer opportunities for data mining and natural language processing to answer questions about quality of care, key aspects of resident education, or attributes of the residents' learning environment.

Objective

We used data obtained from the EHR to report on inpatient documentation practices of residents and attending physicians at a large academic medical center.

Methods

We conducted a retrospective observational study of deidentified patient notes entered over 7 consecutive months by a multispecialty university physician group at an urban hospital. A novel automated data mining technology was used to extract patient note–related variables.

Results

A sample of 26 802 consecutive patient notes was analyzed using the data mining and modeling tool Healthcare Smartgrid. Residents entered most of the notes (33%, 8178 of 24 787) between noon and 4 pm and 31% (7718 of 24 787) of notes between 8 am and noon. Attending physicians placed notes about teaching attestations within 24 hours in only 73% (17 843 of 24 443) of the records. Surgical residents were more likely to place notes before noon (P < .001). Nonsurgical faculty were more likely to provide attestation of resident notes within 24 hours (P < .001).

Conclusions

Data related to patient note entry was successfully used to objectively measure current work flow of resident physicians and their supervising faculty, and the findings have implications for physician oversight of residents' clinical work. We were able to demonstrate the utility of a data mining model as an assessment tool in graduate medical education.

Introduction

The use of electronic health records (EHRs) in the delivery of patient care is steadily increasing; institutions are making significant investments in EHRs, and payers are providing incentives to promote the use of health information technology.1 This is leading to the broad implementation of EHR systems and the phasing out of paper records in private practice settings, urgent care centers, and hospitals.

The EHR has been shown to improve delivery of patient care, increase productivity, and boost billing efficiency.24 In addition, EHR data have been processed using data mining and natural language methods to answer patient care–related questions5,6 and to identify system pitfalls in health care delivery and graduate medical education.7,8 

In our study, we used a new EHR data mining methodology to objectively measure resident and attending note-entry practices at a large academic medical center.

Materials and Methods

To collect the data, an automatic system was used to feed inpatient clinical data (patient encounter notes) from an urban academic hospital's EHR system onto a secure server belonging to a 600-physician university-based multispecialty practice group involved in training medical and surgical residents. An EHR has been used at the institution for more than a decade, and paper patient records have been phased out. We conducted a retrospective observational study of deidentified patient notes during 7 continuous months in 2011 and early 2012. The deidentification included excluding and scrambling protected health information, which was not accessible to any of the investigators or other personnel involved in the study.

The study received approval from Wayne State University's Internal Review Board.

Notes were automatically analyzed by a commercially available data mining and modeling tool, Healthcare Smartgrid (ProcessProxy Corp). This is a patented method of process mining known as Process Arbitrage, which is normally used by the university physician practice for quality improvement purposes.9,10 The data mining tools allowed for the automatic extraction of fields of interest, which included the month of the encounters, the time of resident note entry, the time of required attending attestations (in the form of cosignatures and any needed corrections), and the medical specialty involved. We divided the 24-hour day into six 4-hour time periods and calculated the frequency of resident notes in each period. We also categorized the time interval between note completion and attending physician completion of the attestation into 5 categories—within 24 hours, 24 hours to 1 week, 1 week to 1 month, 1 month to 1 year, and more than 1 year—and calculated the number of attestations in each category. Finally, we recorded the month of the academic year during which the note was created to assess for any change in pattern due to familiarity and efficiency that may occur later on in the year. We used the Pearson χ2 test to perform comparisons between categorical variables using the statistical package SAS version 9.3 (SAS Institute Inc).

Results

The automated system identified 26 802 patient encounter notes in the form of progress notes and consultation reports entered between August 2011 and February 2012. We excluded 2015 notes with missing data, resulting in 24 787 notes that were analyzed. Residents created notes throughout the 24-hour day. However, most notes (33%, 8178 of 24 787) were entered between noon and 4 pm, and 31% (7718 of 24 787) were entered between 8 am and noon (figure 1). The time residents placed notes into the EHR did not significantly change over the academic year for the 9 specialties with the largest number of EHR entries (figure 2).

FIGURE 1

Total Number of Notes Entered by Residents During the 6 Designated Time Periods

FIGURE 1

Total Number of Notes Entered by Residents During the 6 Designated Time Periods

FIGURE 2

Resident Note Placement Patterns

Percentage of notes placed by the top 9 specialties per time period in August (A) and in January (B). Period 1 (00:01 to 04:00); period 2 (04:01 to 08:00); period 3 (08:01 to 12:00); period 4 (12:01 to 16:00); period 5 (16:01 to 20:00); period 6 (20:01 to 24:00).

FIGURE 2

Resident Note Placement Patterns

Percentage of notes placed by the top 9 specialties per time period in August (A) and in January (B). Period 1 (00:01 to 04:00); period 2 (04:01 to 08:00); period 3 (08:01 to 12:00); period 4 (12:01 to 16:00); period 5 (16:01 to 20:00); period 6 (20:01 to 24:00).

Notes were further analyzed based on specialty type. Surgical specialties were obstetrics and gynecology, neurological surgery, ophthalmology, otolaryngology, cardiothoracic surgery, general surgery services, plastic surgery, vascular surgery, and urology; nonsurgical specialties were family medicine, internal medicine and its subspecialties, dermatology, neurology, physical medicine and rehabilitation, psychiatry, and radiation oncology. Surgical residents were more likely to create notes before noon than nonsurgical residents, and surgical residents created about 31% (1756 of 5746) of their notes between midnight and 8 am (P < .001; figure 3A).

FIGURE 3

Documentation by Specialty Type

(A) Surgical residents are more likely to place notes before noon (P < .001).

(B) Nonsurgical supervising attending physicians are more likely to place attending attestations within 24 hours (P < .001).

FIGURE 3

Documentation by Specialty Type

(A) Surgical residents are more likely to place notes before noon (P < .001).

(B) Nonsurgical supervising attending physicians are more likely to place attending attestations within 24 hours (P < .001).

Attending physicians placed approximately 73% (17 843 of 24 443) of their teaching attestations within 24 hours of the encounter. Attestations were entered more than 24 hours after the residents' entry for 27% (6600 of 24 443) of the resident notes. Nonsurgical attendings were more likely to attest resident notes within 24 hours than surgical attendings (P < .001; figure 3B).

Discussion

We used objective measures to assess resident and attending documentation practices in an inpatient setting using data mining methodology. We found that most resident notes were entered in the EHR after 12 pm, and 23% (5778 of 24 787) were entered after 4 pm. This gives insight into the continuous work flow in the inpatient setting that is contrary to the traditional view of trainees' morning work flow. We also found significant delays in documentation and in correction of trainee encounter notes by attending physicians. These attestations confirm the documentation of residents' clinical findings, impressions, and patient management plans. Delays in resident note generation and subsequent attending attestations may negatively affect the efficiency and safety of care.

Residents created 36% (8891 of 24 787) of their notes outside normal working hours, and this did not change with the advance of the academic year. This can potentially infringe on resident duty hours. Although the duty hour standards include time for documentation of care, residents may not properly account for the time spent on EHR data entry.11 New approaches for teaching documentation should include appropriate and efficient EHR use.

We compared EHR entry in surgical and nonsurgical specialties. We found that surgical residents are prompt in their note entry; however, their supervisors often are not timely with attestations. This highlights the need for specialty-specific education and policy regarding the handling of this “indirect care” care domain.

Our study has several limitations. The schedules of the residents and the attendings were not included in the analysis, and some of the after-hour note entry may have involved residents on call or night float. Our study was conducted at a single site, limiting the ability to generalize from the findings.

Conclusion

We reviewed more than 20 000 patient records using data mining to objectively assess resident and attending physician work flow in the new EHR era. Early recognition of the challenges that result from the use of the EHR in teaching settings will allow for implementation of guidelines to generate effective notes with shorter time lags. Future studies should assess the benefit of these interventions on patient care, patient safety, and resident learning.

References

References
1
Steinbrook
R
.
Health care and the American Recovery and Reinvestment Act
.
N Engl J Med
.
2009
;
360
(
11
):
1057
1060
.
2
Brotzman
GL
,
Guse
CE
,
Fay
DL
,
Schellhase
KG
,
Marbella
AM
.
Implementing an electronic medical record at a residency site: physicians' perceived effects on quality of care, documentation, and productivity
.
WMJ
.
2009
;
108
(
2
):
99
103
.
3
Bloom
MV
,
Huntington
MK
.
Faculty, resident, and clinic staff's evaluation of the effects of EHR implementation
.
Fam Med
.
2010
;
42
(
8
):
562
566
.
4
McGuire
MJ
,
Noronha
G
,
Samal
L
,
Yeh
HC
,
Crocetti
S
,
Kravet
S
.
Patient safety perceptions of primary care providers after implementation of an electronic medical record system
.
J Gen Intern Med
.
2013
;
28
(
2
):
184
192
.
5
Jensen
PB
,
Jensen
LJ
,
Brunak
S
.
Mining electronic health records: towards better research applications and clinical care
.
Nat Rev Genet
.
2012
;
13
(
6
):
395
405
.
6
Murff
HJ
,
FitzHenry
F
,
Matheny
ME
,
Gentry
N
,
Kotter
KL
,
Crimin
K
,
et al.
Automated identification of postoperative complications within an electronic medical record using natural language processing
.
JAMA
.
2011
;
306
(
8
):
848
855
.
7
Hripcsak
G
,
Vawdrey
DK
,
Fred
MR
,
Bostwick
SB
.
Use of electronic clinical documentation: time spent and team interactions
.
J Am Med Inform Assoc
.
2011
;
18
(
2
):
112
117
.
8
Thornton
JD
,
Schold
JD
,
Venkateshaiah
L
,
Lander
B
.
Prevalence of copied information by attendings and residents in critical care progress notes
.
Crit Care Med
.
2013
;
41
(
2
):
382
388
.
9
Brennan
JM
,
Peterson
ED
,
Messenger
JC
,
Rumsfeld
JS
,
Weintraub
WS
,
Anstrom
KJ
,
et al.
Linking the National Cardiovascular Data Registry CathPCI Registry with Medicare claims data: validation of a longitudinal cohort of elderly patients undergoing cardiac catheterization
.
Circ Cardiovasc Qual Outcomes
.
2012
;
5
(
1
):
134
140
.
10
Veluswamy
R
.
Golden nuggets: clinical quality data mining in acute care
.
Physician Exec
.
2008
;
34
(
3
):
48
53
.
11
Schlueter
ME
,
Phan
PH
,
Martin
CS
,
Breece
D
,
Boysen
DA
.
Understanding Accreditation Council for Graduate Medical Education (ACGME) guidelines: resident and program director interpretation of work-hour restrictions
.
J Surg Educ
.
2009
;
66
(
6
):
374
378
.

Author notes

Ramesh Madhavan, MD, DM, is Associate Professor, Department of Neurology, Wayne State University; Chi Tang, MD, is Medical Resident, Department of Medicine, Unity Health System; Pratik Bhattacharya, MD, MPH, is Assistant Professor, Department of Neurology, Wayne State University; Fadi Delly, MD, is Neurology Resident, Department of Neurology, Wayne State University; and Maysaa M. Basha, MD, is Assistant Professor, Department of Neurology, Wayne State University.

Funding: The authors report no external funding source for this study.

Conflict of Interest: Dr Madhavan is a consultant with Process Proxy Corporation in a Telemedicine and Advanced Technology Research Center–funded project entitled “Machine Learning-Based Multi-Tasking Management Solution for Civilian and Military Health Systems.” All other authors declare they have no competing interests.