Background 

Advances in information technology have increased remote access to the electronic health record (EHR). Concurrently, standards defining appropriate resident supervision have evolved. How often and under what circumstances inpatient attending physicians remotely access the EHR for resident supervision is unknown.

Objective 

We described a model of attending remote EHR use for resident supervision, and quantified the frequency and magnitude of use.

Methods 

Using a mixed methods approach, general medicine inpatient attendings were surveyed and interviewed about their remote EHR use. Frequency of use and supervisory actions were quantitatively examined via survey. Transcripts from semistructured interviews were analyzed using grounded theory to identify codes and themes.

Results 

A total of 83% (59 of 71) of attendings participated. Fifty-seven (97%) reported using the EHR remotely, with 54 (92%) reporting they discovered new clinical information not relayed by residents via remote EHR use. A majority (93%, 55 of 59) reported that this resulted in management changes, and 54% (32 of 59) reported making immediate changes by contacting cross-covering teams. Six major factors around remote EHR use emerged: resident, clinical, educational, personal, technical, and administrative. Attendings described resident and clinical factors as facilitating “backstage” supervision via remote EHR use.

Conclusions 

In our study to assess attending remote EHR use for resident supervision, attendings reported frequent remote use with resulting supervisory actions, describing a previously uncharacterized form of “backstage” oversight supervision. Future work should explore best practices in remote EHR use to provide effective supervision and ultimately improve patient safety.

What was known and gap

There is interest in enhancing resident supervision; to date no studies have explored supervision via remote use of the electronic health record (EHR).

What is new

A study analyzed factors in attending physicians' remote EHR access and use for supervision and clinical management.

Limitations

Single site study limits generalizability; survey tool without validity evidence.

Bottom line

Remote supervision via attending access of the EHR could offer an added tool for resident supervision to improve resident education and patient safety.

The clinical environment in which residents learn has undergone tremendous change over the past decade. In particular, the widespread implementation of the electronic health record (EHR), in part due to federal incentives, has transformed access to patient information.1,2  Prior studies show that residents frequently view the EHR from home to assess clinical information, order tests, and contact cross-cover teams to make changes in clinical management.3  Modern EHRs provide physicians unprecedented access to view clinical information remotely or outside of the immediate patient care setting. Remote EHR access likely affects how residents and their supervising attending physicians interact in the inpatient setting while caring for patients.

Changes in resident work hour limits and supervision requirements over the past decade provide additional context to view the impact of remote EHR access. The 2011 Accreditation Council for Graduate Medical Education (ACGME) duty hour standards included the first explicit requirements for resident supervision in postgraduate training, highlighting the importance of progressive independence and specifically defining levels of supervision as direct, indirect, or oversight.4  Supervision continues to be a priority in the ACGME's Next Accreditation System, established in 2012.5 

Little attention has been paid to if or how the EHR affects attending supervision of residents, or how often attendings use it for resident supervision. Interestingly, senior residents have been reported to value remote EHR access as a facilitator in developing trust and providing supervision to interns.6  Studies on the effects of computerized provider order entry suggested a role for supervision, and commentaries have postulated that supervision may be enhanced by thoughtful EHR use.79  No study to date has investigated the implications of attending remote EHR use on resident supervision and patient care.

To address this gap, we examined how attendings remotely access EHR on inpatient general medicine teaching services. We define remote access as use of the EHR away from the resident teams and the immediate patient care setting. This study has 2 aims: (1) to identify attending patterns of EHR use, including overall frequency of remote access, tasks performed, how use varies by day of call cycle, and frequency of supervisory actions; and (2) to describe a conceptual model to identify factors influencing why attendings remotely access the EHR, and elicit their perspectives on supervision occurring as a result of information discovered via remote EHR access.

Setting

The University of Chicago general medicine service has 4 teams consisting of 1 attending physician (internist, hospitalist, subspecialist, or chief resident), 1 resident (postgraduate year 2 [PGY-2] or higher), and 2 interns (PGY-1). Attendings and residents complete 2- to 4-week rotations. Residents take admitting call every fourth night for 28-hour overnight shifts, while interns alternate day (7 am to 7 pm) and night (7 pm to 7 am) call shifts. Attendings round with the team postcall on new admissions and daily on hospitalized patients. The EpicCare (Epic Systems Corp, Verona, WI) EHR was implemented in 2010 with remote access capability via a secured virtual environment.

Mixed Methods Approach

We used 2 concurrent data collection approaches: a survey and in-person interviews, with a triangulation design, in which we interpreted our 2 datasets together to address the research questions.10  Mixed methods are used under a pragmatist paradigm to provide multiple perspectives into a poorly described phenomenon. This allowed for a richer and more comprehensive view of our research questions, using all practical means to obtain knowledge and compensating for some of the limitations of both methods.1012  Our research questions corresponded to the aims of the study: (1) when and how often do attendings remotely access the EHR, for what specific tasks, and how often do they perform supervisory actions; and (2) why do attendings remotely access the EHR, and how do they perceive this relates to resident supervision? Quantitative methods assessed the magnitude and frequency of remote EHR access and supervisory actions.11  Qualitative methods probed the reasons behind attendings' remote EHR access and how it related to resident supervision.11,12 

Sampling Strategy

A purposive sampling strategy was used to obtain data from stakeholders directly involved in the phenomenon of interest.1214  We approached general medicine service attendings within 1 to 2 weeks following their rotation to minimize recall bias. Subsequent data collection was guided by theoretical sampling, informed by ongoing iterative data analysis to determine theoretical saturation.1214  Data were collected between January and November 2012, to sample a range of participants over time to capture a description of the phenomenon independent of seasonal changes and resident experience.

Data Collection

Participants provided oral consent and completed both a survey and interview. The 31-item, paper-based survey collected demographic information, and measured timing and frequency of EHR use, tasks performed, and frequency of supervisory actions. The survey was developed from prior work on resident supervision, and was informed by literature and expert review.15  It was pilot-tested with recent residency graduates, and revised based on these discussions. Thirty-minute, semistructured interviews to explore attending remote EHR use, clinical decisions and events related to remote use, and use for resident supervision were conducted by 1 investigator (S.K.M.), and were digitally recorded. The interview script was developed based on expert discussion and a literature review. We used critical incident technique to solicit specific examples of clinical events during the rotation that were influenced by attendings' remote use of the EHR. This allows rare events to be documented, and has been used in previous qualitative work on resident supervision to explore specific patient care interactions.16,17 

The University of Chicago Institutional Review Board approved this study.

Data Analysis

We analyzed survey data using Stata version 13.1 (StataCorp LLP, College Station, TX). We used descriptive statistics to report frequencies, and 2-sample tests of proportion as appropriate to examine differences in EHR use by task across different days in the admitting call cycle, to test the association between frequency of use for specific tasks and days of the call cycle that represented distinct points in the care of a patient (eg, postcall versus on-call day) and different contexts of EHR use (eg, in-house versus remote use).

We performed a qualitative analysis of interview transcripts with a grounded theory approach using the constant comparative method to develop themes describing factors that influence attending remote EHR use.1214  Recorded interviews were transcribed and anonymized. Using ATLAS.ti 7.1 (ATLAS.ti Scientific Software Development GmbH, Berlin, Germany), 3 investigators (S.K.M., K.T., J.M.F.) coded an initial number of transcripts, with sentences and phrases as units of analysis. We used an inductive, iterative process to identify themes that encompassed several codes. Memo-writing and diagrams were used to facilitate analysis.14  Investigators met at regular intervals to establish a coding framework and resolve discrepancies via discussion until consensus was achieved. New codes that emerged were included in the coding scheme until thematic saturation was reached.14  The coding framework was applied to all transcripts.

Trustworthiness was enhanced by peer scrutiny, review of design and implementation, and regular investigator meetings.18  Reflexivity was maintained by considering researcher characteristics of the investigators, who at the time were a research fellow (S.K.M.), medical student (K.T.), and faculty members (D.O.M., V.M.A., J.M.F.) at the study institution. This study adheres to the criteria established by the standards for reporting qualitative research.19 

Of 71 eligible attendings, 59 (83%) participated in the study, completing both the survey and interview. This included attendings with multiple service rotations, and there were 45 unique attending participants (figure 1). The majority were female (58%, 26 of 45), and 69% (31 of 45) were general internists. About 25% (11 of 45) had completed postgraduate training within the past 3 years, 42% (19) within the past 4 to 15 years, and 36% (16) more than 15 years earlier. Most (80%, 36 of 45) spent at least 4 weeks on inpatient service per year.

figure 1

Attending Participation and Eligibility

a Attending blocks are defined as 2 weeks of service per month. Four attendings are scheduled every block, and there are 2 blocks per month in the calendar year.

figure 1

Attending Participation and Eligibility

a Attending blocks are defined as 2 weeks of service per month. Four attendings are scheduled every block, and there are 2 blocks per month in the calendar year.

Close modal

Quantitative Analysis

Nearly all attendings (96%, 57 of 59) used the EHR remotely. Most (93%, 55 of 59) estimated they used the EHR for 60 to 90 minutes daily, and a small number reported use for more than 90 minutes (7%, 4 of 59).

Attendings were surveyed on how often they used the EHR for specific tasks. The most common tasks were completing documentation, monitoring clinical information, and reviewing consultant notes (table 1). Attendings reported both in-house and remote EHR use, and specified use for each task by day of the admitting call cycle. For in-house EHR use, attendings reviewed past notes/history more frequently on postcall days (75%, 44 of 59) versus on-call days (54%, 32 of 59; z = 2.31; P = .021). For remote EHR use, attendings monitored clinical information more frequently on postcall (81%, 48 of 59) versus on-call days (64%, 38 of 59; z = 2.07; P = .038). They also reviewed consultant notes more frequently on postcall (76%, 45 of 59) versus on-call days (56%, 33 of 59; z = 2.33; P = .020).

table 1

Frequency of Attending-Reported Electronic Health Record (EHR) Tasks by Day in Call Cyclea

Frequency of Attending-Reported Electronic Health Record (EHR) Tasks by Day in Call Cyclea
Frequency of Attending-Reported Electronic Health Record (EHR) Tasks by Day in Call Cyclea

In response to the question of how often remote EHR use led to supervisory actions with the resident team, nearly all attendings (93%, 55 of 59) reported using it to confirm clinical information received from residents, and 54% (32 of 59) reported that they did this often (at least 3 times per week during the attending's preceding rotation). Almost every attending (92%, 54 of 59) reported discovering information that residents did not relay adequately, with 25% (15 of 59) noting that this occurred often, and 93% (55 of 59) reported making changes in clinical management as a result of these discoveries, with 20% (12 of 59) reporting it happened often. Most (86%, 51 of 59) reported that management changes occurred the following day on rounds, and more than half (54%, 32 of 59) reported immediate changes in clinical care as a result of discovering information. This was defined as a call from home to the cross-covering team, with 14% (8 of 59) reporting that this occurred 3 times per week.

Qualitative Analysis

Six themes for factors influencing remote EHR use emerged from the qualitative analysis: (1) resident factors; (2) clinical factors; (3) educational factors; (4) personal factors; (5) technical factors; and (6) administrative factors (table 2).

table 2

Factors Influencing Attending Remote Access of the Electronic Health Record (EHR)

Factors Influencing Attending Remote Access of the Electronic Health Record (EHR)
Factors Influencing Attending Remote Access of the Electronic Health Record (EHR)

The majority of codes related to resident and clinical factors (provided as online supplemental material). Attendings often referred to remote EHR access as a tool in determining entrustment, and adjusted use as needed based on perceptions of resident competence: “I'm going to be like Ronald Reagan—I'm going to trust but verify. I will randomly go on and check some things, and I'm not going to say anything if things are fine.” (Attending MM, interview 153; theme: resident factors; subtheme: trust but verify)

Remote access was described as helpful in providing supervision in a dynamic clinical environment:

“The model that we typically have for supervision is you check in once, maybe twice a day with your team after rounds. But there's . . . stuff changing all the time, and they're reacting to that information, making decisions based on the information that they have when it comes to them.” (Attending SS, interview 100; theme: resident factors; subtheme: supervising residents in the event of evolving patient information or uncertainty)

Remote EHR access was also valued as providing a safety net for patient care, particularly in complex or uncertain clinical cases. “With how sick our patients are and to know our residents are in training, we can't expect that they would see all of it, so I do feel that monitoring is appropriate.” (Attending A, interview 101; theme: resident factors; subtheme: acting as resident safety net)

Work hours and handoffs were frequently discussed, with remote use described as a means to maintain continuity of care:

“As the attending, you're the link, because now with work hours and days off, you may be the only person who's seen them from start to finish . . . the record makes me feel like I know the patient more, because it's not like I'm just hearing from other people, I can really follow everything real-time.” (Attending R, interview 129; theme: clinical factors; subtheme: more active attending role to maintain continuity of care)

Finally, attendings noted reasons for remote EHR use unrelated to residents or supervision, such as personal curiosity, their own experience or comfort, convenience, and use for administrative tasks (table 2).

A conceptual model for attending remote use of the EHR emerged from the quantitative and qualitative analyses (figure 2). It integrated the reasons that attendings described for remote use, along with the supervisory actions that were most frequently reported.

figure 2

Conceptual Model—Factors Influencing Attending Remote Electronic Health Record (EHR) Use and Resulting Supervisory Actions

figure 2

Conceptual Model—Factors Influencing Attending Remote Electronic Health Record (EHR) Use and Resulting Supervisory Actions

Close modal

In this study, attendings frequently accessed the EHR remotely and discovered information outside of the immediate patient care setting. As a result of their remote EHR use, nearly all attendings reported making changes in patient care plans based on clinical information discovered. Attendings accessed the EHR remotely for reasons commonly related to residents and clinical care. Many subthemes emerged, suggesting use for resident supervision.

Attendings directly sought clinical information on patients more commonly via remote access on the postcall day following admission. This finding could suggest (as was noted in the interviews) that attendings use the EHR more actively when patients are more likely to be undifferentiated, and when residents may require more attending involvement or supervision. Interestingly, attendings noted higher overall and remote EHR use on postcall days in all but 1 activity (preparing teaching points for rounds). Remote EHR access may be a tool attendings use to monitor real-time clinical decision-making and management efficacy at the moment patient information is available.

In interviews, attendings frequently described resident and clinical factors as drivers of remote EHR use. Among the themes and subthemes emerging from qualitative analysis, attendings described remote EHR use as a type of supervision most aptly characterized as backstage oversight.21  In contrast to direct or indirect supervision, this oversight supervision occurs when attendings provide feedback on clinical care provided by residents.5  Backstage oversight, defined by the framework on which the ACGME definitions were based, is supervision of which the resident may not be directly aware.21  A prominent subtheme was the concept of remote use as a safety net for clinical care. The changing paradigm of resident supervision has been influenced by the heightened focus on quality and patient safety over the last decade.22  Much of the focus on the EHR has been through the Centers for Medicare and Medicaid Services Meaningful Use program, calling for the EHR to be utilized in a manner that improves quality, safety, and clinical outcomes.23  If attendings are using backstage oversight via the EHR for patient safety purposes as this study suggests, the EHR could be further leveraged to contribute to this effort by highlighting meaningful supervisory use in postgraduate training.

Attendings discussed their remote use of the EHR through the lens of resident trust. Entrustment is a complex process for which supervisors must make specific, personalized decisions regarding the ability to trust residents with certain tasks.24  Five major determinants of entrustment (trainee, supervisor, relationship, task, and contextual factors) have been well described, and are well aligned with the findings of our study.25  Remote use of the EHR may further inform attending judgments regarding entrustment of autonomy to residents, particularly for ad hoc entrustment decisions.25 

Time constraints were a prominent reason influencing attending remote EHR access in several subthemes. Work hour limitations have significantly affected the time spent in direct contact between supervising attendings and residents.26  Remote EHR access may help mitigate the effects of limited contact between supervising attendings and residents.

Our study has limitations. It was performed at a single site, academic tertiary care center with an EHR equipped with the capability for remote access, which may limit generalizability. Our sampling was restricted to internal medicine attendings within 1 inpatient service. Answers were self-reported, and nonresponse or recall bias may have affected results. Finally, we did not obtain resident viewpoints on supervision, an important perspective to consider, and our study did not assess the degree to which changes in management were communicated to residents to allow them to learn from their decisions.

Future work in this area should identify best practices to develop and refine attending remote EHR access as a method of providing resident supervision. Alternative methods of examining the EHR should be utilized to further study this question. For example, data mining and natural language processing of clinical documentation are techniques used in both quality improvement and medical education that could be applied to further study the role of the EHR in resident supervision.27,28 

We found that attendings remotely access the EHR on a frequent basis, and often make changes in clinical care as a result, in a manner consistent with backstage oversight supervision. Attendings report different reasons for remote EHR use, with the most commonly described centering on resident and clinical factors. These insights into how attendings use the EHR can help support efforts toward development of appropriate supervisory techniques and ultimately improve resident supervision and patient safety.

1
Blumenthal
D.
Launching
HITECH.
N Engl J Med
.
2010
;
362
(
2
):
382
385
.
2
Nguyen
L,
Bellucci
E,
Nguyen
LT.
Electronic health records implementation: an evaluation of information system impact and contingency factors
.
Int J Med Inform
.
2014
;
83
(
11
):
779
796
.
3
Deaño
RC,
DeKosky
A,
Appannagari
A,
et al.
Resident time spent in clinical and educational activities at home: implications for duty hours
.
Arch Intern Med
.
2011
;
171
(
11
):
1038
1039
.
4
ACGME Task Force on Quality Care and Professionalism
.
The ACGME 2011 duty hour standards: enhancing quality of care, supervision, and resident professional development
.
2017
.
5
Nasca
TJ,
Philibert
I,
Brigham
T,
et al.
The next GME accreditation system—rationale and benefits
.
N Engl J Med
.
2012
;
366
(
11
):
1051
1056
.
6
Sheu
L,
O'Sullivan
PS,
Aagaard
EM,
et al.
How residents develop trust in interns: a multi-institutional mixed methods study
.
Acad Med
.
2016
;
91
(
10
):
1406
1415
.
7
Wong
B,
Kuper
A,
Robinson
N,
et al.
Computerised provider order entry and residency education in an academic medical centre
.
Med Educ
.
2012
;
46
(
8
):
795
806
.
8
Brickner
L,
Clement
M,
Patton
M.
Resident supervision and the electronic medical record
.
Arch Intern Med
.
2008
;
168
(
10
):
1117
1118
.
9
Peled
JU,
Sagher
O,
Morrow
JB,
et al.
Do electronic health records help or hinder medical education?
PLoS Med
.
2009
;
6
(
5
):
e1000069
.
10
Creswell
JW,
Plano Clark VL. Designing and Conducting Mixed Methods Research. 2nd ed
.
Thousand Oaks, CA
:
SAGE Publications Inc;
2011
.
11
O'Brien
BC,
O'Sullivan
PS.
Mixed methods in health professions education scholarship
.
J Grad Med Educ
.
2017
;
9
(
2
):
253
254
.
12
Teherani
A,
Martimianakis
T,
Stenfors-Hayes
T,
et al.
Choosing a qualitative research approach
.
J Grad Med Educ
.
2015
;
7
(
4
):
669
670
.
13
Wright
S,
O'Brien
BC,
Nimmon
L,
et al.
Research design considerations
.
J Grad Med Educ
.
2016
;
8
(
1
):
97
98
.
14
Watling
CJ,
Lingard
L.
Grounded theory in medical education research: AMEE Guide No. 70
.
Med Teach
.
2012
;
34
(
10
):
850
861
.
15
Farnan
JM,
Johnson
JK,
Meltzer
DO,
et al.
On-call supervision and resident autonomy: from micromanager to absentee attending
.
Am J Med
.
2009
;
122
(
8
):
784
788
.
16
Flanagan
JC.
The critical incident technique
.
Psychol Bull
.
1954
;
51
(
4
):
327
358
.
17
Farnan
JM,
Johnson
JK,
Meltzer
DO,
et al.
Resident uncertainty in clinical medical decision making and impact on patient care: a qualitative study
.
Qual Saf Health Care
.
2008
;
17
(
2
):
122
126
.
18
Shenton
AK.
Strategies for ensuring trustworthiness in qualitative research projects
.
Educ Inform
.
2004
;
22
(
2
):
63
75
.
19
O'Brien
BC,
Harris
IB,
Beckman
TJ,
et al.
Standards for reporting qualitative research: a synthesis of recommendations
.
Acad Med
.
2014
;
89
(
9
):
1245
1251
.
20
Verghese
A.
Culture shock—patient as icon, icon as patient
.
N Engl J Med
.
2008
;
359
(
26
):
2748
2751
.
21
Kennedy
TJ,
Lingard
L,
Baker
GR,
et al.
Clinical oversight: conceptualizing the relationship between supervision and safety
.
J Gen Intern Med
.
2007
;
22
(
8
):
1080
1085
.
22
Halpern
SD,
Detsky
AS.
Graded autonomy in medical education—managing things that go bump in the night
.
N Engl J Med
.
2014
;
370
(
12
):
1086
1089
.
23
Blumenthal
D,
Tavenner
M.
The “meaningful use” regulation for electronic health records
.
N Engl J Med
.
2010
;
363
(
6
):
501
504
.
24
Martin
SK,
Farnan
JM,
Flores
A,
et al.
Exploring entrustment: housestaff autonomy and patient readmission
.
Am J Med
.
2014
;
127
(
8
):
791
797
.
25
ten Cate
O,
Hart
D,
Ankel
F,
et al.
Entrustment decision making in clinical training
.
Acad Med
.
2016
;
91
(
2
):
191
198
.
26
Holmboe
ES,
Ward
DS,
Reznick
RK,
et al.
Faculty development in assessment: the missing link in competency-based medical education
.
Acad Med
.
2011
;
86
(
4
):
460
467
.
27
Madhavan
R,
Tang
C,
Bhattacharya
P,
et al.
Evaluation of documentation patterns of trainees and supervising physicians using data mining
.
J Grad Med Educ
.
2014
;
6
(
3
):
577
580
.
28
Denny
JC,
Spickard
A
III,
Speltz
PJ,
et al.
Using natural language processing to provide personalized learning opportunities from trainee clinical notes
.
J Biomed Inform
.
2015
;
56
:
292
299
.

Author notes

Funding: This study was supported in part by an NIH T32 Health Services Research Training Grant (NIH 5T32HS000084-13).

Competing Interests

Conflict of interest: The authors declare they have no competing interests.

Preliminary results were presented as an oral abstract at the Society of General Internal Medicine Midwest Regional Meeting, Chicago, Illinois, September 2012; and as a poster at the Society of Hospital Medicine Annual Meeting, National Harbor, Maryland, March 2013.

The authors would like to thank the Medical Education Research, Innovation, Teaching, and Scholarship (MERITS) program at the University of Chicago Pritzker School of Medicine for assistance with this work.

Editor's Note: The online version of this study contains a table of resident and clinical factors influencing attending remote access of the electronic health record.

Supplementary data