The electronic health record (EHR) includes a rich data set that may offer opportunities for data mining and natural language processing to answer questions about quality of care, key aspects of resident education, or attributes of the residents' learning environment.
We used data obtained from the EHR to report on inpatient documentation practices of residents and attending physicians at a large academic medical center.
We conducted a retrospective observational study of deidentified patient notes entered over 7 consecutive months by a multispecialty university physician group at an urban hospital. A novel automated data mining technology was used to extract patient note–related variables.
A sample of 26 802 consecutive patient notes was analyzed using the data mining and modeling tool Healthcare Smartgrid. Residents entered most of the notes (33%, 8178 of 24 787) between noon and 4 pm and 31% (7718 of 24 787) of notes between 8 am and noon. Attending physicians placed notes about teaching attestations within 24 hours in only 73% (17 843 of 24 443) of the records. Surgical residents were more likely to place notes before noon (P < .001). Nonsurgical faculty were more likely to provide attestation of resident notes within 24 hours (P < .001).
Data related to patient note entry was successfully used to objectively measure current work flow of resident physicians and their supervising faculty, and the findings have implications for physician oversight of residents' clinical work. We were able to demonstrate the utility of a data mining model as an assessment tool in graduate medical education.
The use of electronic health records (EHRs) in the delivery of patient care is steadily increasing; institutions are making significant investments in EHRs, and payers are providing incentives to promote the use of health information technology.1 This is leading to the broad implementation of EHR systems and the phasing out of paper records in private practice settings, urgent care centers, and hospitals.
The EHR has been shown to improve delivery of patient care, increase productivity, and boost billing efficiency.2–4 In addition, EHR data have been processed using data mining and natural language methods to answer patient care–related questions5,6 and to identify system pitfalls in health care delivery and graduate medical education.7,8
In our study, we used a new EHR data mining methodology to objectively measure resident and attending note-entry practices at a large academic medical center.
Materials and Methods
To collect the data, an automatic system was used to feed inpatient clinical data (patient encounter notes) from an urban academic hospital's EHR system onto a secure server belonging to a 600-physician university-based multispecialty practice group involved in training medical and surgical residents. An EHR has been used at the institution for more than a decade, and paper patient records have been phased out. We conducted a retrospective observational study of deidentified patient notes during 7 continuous months in 2011 and early 2012. The deidentification included excluding and scrambling protected health information, which was not accessible to any of the investigators or other personnel involved in the study.
The study received approval from Wayne State University's Internal Review Board.
Notes were automatically analyzed by a commercially available data mining and modeling tool, Healthcare Smartgrid (ProcessProxy Corp). This is a patented method of process mining known as Process Arbitrage, which is normally used by the university physician practice for quality improvement purposes.9,10 The data mining tools allowed for the automatic extraction of fields of interest, which included the month of the encounters, the time of resident note entry, the time of required attending attestations (in the form of cosignatures and any needed corrections), and the medical specialty involved. We divided the 24-hour day into six 4-hour time periods and calculated the frequency of resident notes in each period. We also categorized the time interval between note completion and attending physician completion of the attestation into 5 categories—within 24 hours, 24 hours to 1 week, 1 week to 1 month, 1 month to 1 year, and more than 1 year—and calculated the number of attestations in each category. Finally, we recorded the month of the academic year during which the note was created to assess for any change in pattern due to familiarity and efficiency that may occur later on in the year. We used the Pearson χ2 test to perform comparisons between categorical variables using the statistical package SAS version 9.3 (SAS Institute Inc).
The automated system identified 26 802 patient encounter notes in the form of progress notes and consultation reports entered between August 2011 and February 2012. We excluded 2015 notes with missing data, resulting in 24 787 notes that were analyzed. Residents created notes throughout the 24-hour day. However, most notes (33%, 8178 of 24 787) were entered between noon and 4 pm, and 31% (7718 of 24 787) were entered between 8 am and noon (figure 1). The time residents placed notes into the EHR did not significantly change over the academic year for the 9 specialties with the largest number of EHR entries (figure 2).
Notes were further analyzed based on specialty type. Surgical specialties were obstetrics and gynecology, neurological surgery, ophthalmology, otolaryngology, cardiothoracic surgery, general surgery services, plastic surgery, vascular surgery, and urology; nonsurgical specialties were family medicine, internal medicine and its subspecialties, dermatology, neurology, physical medicine and rehabilitation, psychiatry, and radiation oncology. Surgical residents were more likely to create notes before noon than nonsurgical residents, and surgical residents created about 31% (1756 of 5746) of their notes between midnight and 8 am (P < .001; figure 3A).
Attending physicians placed approximately 73% (17 843 of 24 443) of their teaching attestations within 24 hours of the encounter. Attestations were entered more than 24 hours after the residents' entry for 27% (6600 of 24 443) of the resident notes. Nonsurgical attendings were more likely to attest resident notes within 24 hours than surgical attendings (P < .001; figure 3B).
We used objective measures to assess resident and attending documentation practices in an inpatient setting using data mining methodology. We found that most resident notes were entered in the EHR after 12 pm, and 23% (5778 of 24 787) were entered after 4 pm. This gives insight into the continuous work flow in the inpatient setting that is contrary to the traditional view of trainees' morning work flow. We also found significant delays in documentation and in correction of trainee encounter notes by attending physicians. These attestations confirm the documentation of residents' clinical findings, impressions, and patient management plans. Delays in resident note generation and subsequent attending attestations may negatively affect the efficiency and safety of care.
Residents created 36% (8891 of 24 787) of their notes outside normal working hours, and this did not change with the advance of the academic year. This can potentially infringe on resident duty hours. Although the duty hour standards include time for documentation of care, residents may not properly account for the time spent on EHR data entry.11 New approaches for teaching documentation should include appropriate and efficient EHR use.
We compared EHR entry in surgical and nonsurgical specialties. We found that surgical residents are prompt in their note entry; however, their supervisors often are not timely with attestations. This highlights the need for specialty-specific education and policy regarding the handling of this “indirect care” care domain.
Our study has several limitations. The schedules of the residents and the attendings were not included in the analysis, and some of the after-hour note entry may have involved residents on call or night float. Our study was conducted at a single site, limiting the ability to generalize from the findings.
We reviewed more than 20 000 patient records using data mining to objectively assess resident and attending physician work flow in the new EHR era. Early recognition of the challenges that result from the use of the EHR in teaching settings will allow for implementation of guidelines to generate effective notes with shorter time lags. Future studies should assess the benefit of these interventions on patient care, patient safety, and resident learning.
Ramesh Madhavan, MD, DM, is Associate Professor, Department of Neurology, Wayne State University; Chi Tang, MD, is Medical Resident, Department of Medicine, Unity Health System; Pratik Bhattacharya, MD, MPH, is Assistant Professor, Department of Neurology, Wayne State University; Fadi Delly, MD, is Neurology Resident, Department of Neurology, Wayne State University; and Maysaa M. Basha, MD, is Assistant Professor, Department of Neurology, Wayne State University.
Funding: The authors report no external funding source for this study.
Conflict of Interest: Dr Madhavan is a consultant with Process Proxy Corporation in a Telemedicine and Advanced Technology Research Center–funded project entitled “Machine Learning-Based Multi-Tasking Management Solution for Civilian and Military Health Systems.” All other authors declare they have no competing interests.