ABSTRACT

Background 

Just-in-time (JIT) training refers to education occurring immediately prior to clinical encounters. An in situ JIT room in a pediatric emergency department (ED) was created for procedural education.

Objective 

We examined trainee self-reported JIT room use, its impact on trainee self-perception of procedural competence/confidence, and the effect its usage has on the need for intervention by supervising physicians during procedures.

Methods 

Cross-sectional survey study of a convenience sample of residents rotating through the ED and supervising pediatric emergency medicine physicians. Outcomes included JIT room use, trainee procedural confidence, and frequency of supervisor intervention during procedures.

Results 

Thirty-one of 32 supervising physicians (97%) and 122 of 186 residents (66%) completed the survey, with 71% of trainees reporting improved confidence, and 68% reporting improved procedural skills (P < .05, +1.4-point average skills improvement on a 5-point Likert scale). Trainees perceived no difference among supervising physicians intervening in procedures with or without JIT room use (P = .30, paired difference −0.0 points). Nearly all supervisors reported improved trainee procedural confidence, and 77% reported improved trainee procedural skills after JIT room use (P < .05, paired difference +1.8 points); 58% of supervisors stated they intervene in procedures without trainee JIT room use, compared with 42% with JIT room use (P < .05, paired difference −0.4 points).

Conclusions 

Use of the JIT room led to improved trainee confidence and supervisor reports of less procedural intervention. Although it carries financial and time costs, an in situ JIT room may be important for convenient JIT training.

What was known and gap

Increased supervision and duty hour restrictions have led to fewer clinical opportunities for trainees, including opportunities to perform procedures in the emergency department.

What is new

Creation of an in-situ just-in-time (JIT) procedural training room in a pediatric emergency department.

Limitations

Single institution study; convenience sample; survey instrument lacks validity evidence.

Bottom line

Use of the JIT procedural training room led to improved trainee confidence and supervisors' reports of fewer interventions during resident procedures.

Introduction

Increased supervision and duty hour restrictions have led to fewer clinical opportunities for trainees. Concurrently, the Accreditation Council for Graduate Medical Education has developed milestones for residents, which include procedural or technical skills under the core competency of patient care. Just-in-time (JIT) procedural training allows for practice of a skill immediately prior to performing a procedure in an effort to maximize trainee learning, confidence, and patient safety. While similar to in situ simulation in its geographic proximity to procedural performance, JIT training adds temporal proximity as well.

A number of studies19  have examined JIT training, yet there are no known studies examining the use of dedicated JIT space to practice common procedures near the clinical space, rather than a simulation center. Simulation-based education, as in the JIT room, can provide a reproducible standard environment for deliberate practice as well as both formative and summative assessment.10  Gaining a better understanding of learners' perspectives can assist educators in the implementation of a specific simulation curriculum.3 

Seattle Children's Hospital opened a new emergency department (ED) in April 2013 that includes a JIT training room, conveniently located within the ED clinical space, and is available 24 hours a day for procedural education in splinting, suturing, and lumbar puncture (LP). This room is used daily for physician and nurse education, but is always available for JIT training. This was thought to be a cost-effective approach. Location within the ED clinical space maximizes accessibility to trainees working clinical shifts and allows for deliberate practice, a structured repetitive training designed to acquire or improve a critical aspect of performance for a specific level of performance or mastery.8,9 

We sought to examine the impact of the JIT room on resident and supervisor procedural experiences in the ED. We hypothesized that the JIT room would be a convenient, cost-effective space for preprocedural training that would potentially improve trainee self-confidence and supervisor confidence in the trainee, and allow for less supervisor intervention during procedures.

Methods

We conducted a cross-sectional survey study using a convenience sample of trainees and supervising physicians performed at a tertiary care pediatric hospital, including family medicine, emergency medicine (EM), and pediatrics residents who worked in the ED, as well as pediatric emergency medicine (PEM) attending physicians and fellows. Medical students and visiting residents were excluded.

Two surveys were administered: 1 to trainees and 1 to PEM supervising physicians. The surveys were developed using expert consultation in the fields of education and PEM. They were pretested by various PEM faculty members at the study institution for content and ease of use. Demographic survey data included years of training/years of posttraining, specialty, prior experience with JIT training, and baseline comfort with suturing, splinting, and LP. Using 5-point Likert scales and divided time percentages, the surveys queried use of the JIT room, subsequent confidence in procedure performance, and supervisor intervention during procedures. Open-ended questions about the JIT room included barriers to use, improvement recommendations, and procedures to add.

The anonymous surveys were disseminated using Research Electronic Data Capture, or REDCap (Vanderbilt University, Nashville, TN), using an e-mail link sent to eligible subjects. The survey was open from June 6 through July 11, 2014. A maximum of 8 reminders were e-mailed inviting subjects to complete the surveys.

The study was granted a review exemption by the Seattle Children's Hospital Institutional Review Board.

Demographic characteristics and JIT room use were summarized. Paired t tests compared responses to contrasting questions to assess use of the JIT room based on trainee self-reported competence, whether usage was associated with higher trainee perceived confidence or trainee procedural skills, and if usage was related to supervisor intervention during procedures. For each t test, the paired difference, its 95% confidence interval, and the P value are reported. P values below .05 were considered significant. Adjustments were not made for multiple testing, as this is an exploratory study. Analyses were performed using Stata version 12 (StataCorp LP, College Station, TX) and Microsoft Excel 2010 (Microsoft Corp, Redmond, WA). Subsequent to the survey, JIT room material costs were estimated as a means of describing generalized feasibility.

Results

Thirty-one of 32 supervising physicians (97%) and 122 of 186 residents (66%) completed the survey. Table 1 shows response rates by resident specialty.

table 1

Specialty and Postgraduate Training Year for Trainee Respondents

Specialty and Postgraduate Training Year for Trainee Respondents
Specialty and Postgraduate Training Year for Trainee Respondents

Residents reported that the JIT room is often used if they feel insufficiently skilled in a procedure (figure 1). Residents felt more proficient when using the JIT room (mean = 4.0 on a Likert scale of 1 to 5) than when not using the JIT room (mean = 2.6; P < .05; paired difference +1.4; table 2). Trainees reported that JIT room use has little effect on supervisor decision to procedurally intervene (mean score = 2.6 for both; P = .30; paired difference −0.04; table 2).

figure 1

Trainee Use of Just-in-Time (JIT) Room

figure 1

Trainee Use of Just-in-Time (JIT) Room

table 2

Impressions of Proficiency and Intervention

Impressions of Proficiency and Intervention
Impressions of Proficiency and Intervention

Among supervisors, 94% (29 of 31) reported that they were more likely to use JIT training for procedural practice with the JIT room within the clinical ED space. All supervisors noted that they use the JIT room if a trainee states that he or she is not proficient in a procedure, but only 19% (6 of 31) would do so if a trainee states that he or she is procedurally proficient. The JIT room is most frequently used to teach suture repair and LP skills and less frequently used to teach splinting: 32% (10 of 31) of supervising physicians used the JIT room more than 10 times to teach suturing, 32% (10 of 31) used it more than 10 times to teach LP, and only 3% (1 of 31) of supervisors used it more than 10 times to teach splinting (figure 2).

figure 2

Supervisor Use of Just-in-Time (JIT) Room

figure 2

Supervisor Use of Just-in-Time (JIT) Room

Supervisors strongly agreed that residents feel more proficient if they use the JIT room (mean = 4.7 on a Likert scale of 1–5) and were neutral on residents' feelings of proficiency when they do not use the JIT room (mean = 2.8 on a Likert scale of 1–5; P < .05; paired difference +1.8; table 2). Supervisors were more inclined to intervene if the JIT room was not used (P < .05; paired difference −0.4; table 2).

Lack of time was reported as a major barrier to JIT room use. While most felt that there was not anything to be improved in the JIT room, some respondents suggested standardizing regular use of the room and scheduled supply restocking. Suggestions for added procedures to the JIT room included central and peripheral line placement, intubation, foreign body removal, bladder catheterization, gastric tube replacement, ultrasound, incision and drainage, chest tube placement, and tracheostomy tube replacement.

The JIT room was conceptualized early in the planning of the new ED to allow for efficient JIT education of trainees prior to procedures within the ED. Suture, splinting, and LP material is recycled and reused. See table 3 for cost estimation of materials. Ongoing costs for the space are part of ED overhead.

table 3

Cost Estimates for Just-in-Time Room Training Materials

Cost Estimates for Just-in-Time Room Training Materials
Cost Estimates for Just-in-Time Room Training Materials

Discussion

Both residents and supervisors reported improved trainee skills and confidence with JIT room use. In addition, supervisors reported that they intervene less in procedures after JIT room use; however, there was no difference seen in supervising faculty intervention in procedures as perceived by trainees.

The educational theory of situated learning proposes that effective learning occurs through workplace context and experiential participation,11  allowing learners and teachers to offer unique perspectives on educational interventions. The JIT room's location in the clinical space creates an optimally situated learning environment, as travel to a simulation center is not possible during a clinical shift. Use of a JIT room located within the clinical space can occur between tasks; however, learners must make time to utilize the room and balance clinical demands.

While JIT learning is well grounded in educational theory, studies of the effect of JIT teaching have produced mixed results. In a study of JIT training in infant LP,12  frequency of procedural success was not affected by JIT training. Other markers of procedural proficiency, however, such as accurate initial placement of central lines, improved after JIT practice.7  This may point to a contextual issue with situated cognition in that learning is inherent to the situation in which it occurs.13  A dedicated procedural training space such as a JIT training room that removes the learner from a distracting clinical environment, while still physically being in the clinical space, might improve trainee procedural success by providing a protected space for JIT deliberate practice.

A majority of residents and supervising physicians felt that trainee procedural confidence and competence improved after use of the JIT room. Trainees did not feel that use of the JIT room reduced supervising physician intervention. In contrast, supervising physicians felt that use of the JIT room reduced their interventions in trainee-conducted procedures. This difference may suggest that supervising physicians perform an informal summative assessment of trainees in the JIT room and therefore believe they intervene less in trainee procedures. Trainees may not perceive this change. Additionally, given their limited procedural opportunities, trainees may not have the perspective to sense a difference in physician supervisor intervention.

The JIT room is primarily utilized for skill training and formative assessment. Trainees practice their procedural skills in the space, but are not explicitly required to demonstrate a certain level of competence before performing a procedure.

Trainee and supervising physicians reported that lack of time was the major barrier to use of the JIT room. Another study1  found this to be a similar barrier for JIT infant LP trainer use, as reported by pediatrics interns. Standardizing the expectation that the room should be used for summative assessment prior to performing a procedure on a real patient may transform use of the room into a standard preprocedure step.

Limitations of this study include that it was a single center study, it used a convenience sample, and the results may not generalize to other settings. Our survey was developed by knowledgeable faculty, but lacks validity evidence, and respondents may not have interpreted questions as intended. The ED at which this study took place is heavily staffed by pediatrics and family medicine residents (more so than EM residents), as reflected in the distribution of trainee survey respondents. Given that these learners inherently spent less time in the ED, family medicine and pediatrics residents may be less confident in their procedural skills than EM residents, potentially making them more likely to use the JIT room. The study's reliance on self-report makes it susceptible to recall and reporting bias. We attempted to avoid reporting bias by keeping surveys anonymous and protected from program leaders. Additionally, survey methodology did not allow us to measure definitive endpoints such as patient satisfaction or procedure outcome. Lastly, the JIT room was built with a new ED, so translating this space to other institutions may not be feasible. However, existing space proximate to the ED could be modified to include dedicated JIT training space.

Future research should include evaluation of the JIT room use relative to the experience level of the trainee, as well as explore the effects of summative JIT assessment on procedural performance and, ultimately, quality and safety of patient care.

Conclusion

A dedicated room near the clinical space may be an efficient, time-saving method to provide JIT education, potentially improve trainee skills and confidence, and decrease supervisor intervention in procedures.

References

References
1
Kamdar
G,
Kessler
DO,
Tilt
L,
et al.
Qualitative evaluation of just-in-time simulation-based learning: the learners' perspective
.
Simul Healthc
.
2013
;
8
(
1
):
43
48
.
2
Weinberg
ER,
Auerbach
MA,
Shah
NB.
The use of simulation for pediatric training and assessment
.
Curr Opin Pediatr
.
2009
;
21
(
3
):
282
287
.
3
Shanks
D,
Wond
RY,
Roberts
JM,
et al.
Use of simulator-based medical procedural curriculum: the learner's perspective
.
BMC Med Educ
.
2010
;
10
:
77
.
4
Mendiratta-Lala
M,
Williams
T,
de Quadros
N,
et al.
The use of a simulation center to improve resident proficiency in performing ultrasound-guided procedures
.
Acad Radiol
.
2010
;
17
(
4
):
535
540
.
5
Hishikawa
S,
Kawano
M,
Tanaka
H,
et al.
Mannequin simulation improves the confidence of medical students performing tube thoracotomy: a prospective, controlled trial
.
Am Surg
.
2010
;
76
(
1
):
73
78
.
6
Faulkner
AR,
Bourgeois
AC,
Bradley
YC,
et al.
Simulation-based educational curriculum for fluoroscopically guided lumbar puncture improves operator confidence and reduces patient dose
.
Acad Radiol
.
2015
;
22
(
5
):
668
673
.
7
Barsuk
JH,
McGaghie
WC,
Cohen
ER,
et al.
Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit
.
Crit Care Med
.
2009
;
37
(
10
):
2697
2701
.
8
Causer
J,
Barach
P,
Williams
AM.
Expertise in medicine: using the expert performance approach to improve simulation training
.
Med Educ
.
2014
;
48
(
2
):
115
123
.
9
Ericsson
KA.
Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains
.
Acad Med
.
2004
;
79
(
suppl 10
):
70
81
.
10
Beeson
MS,
Vazenilek
JA.
Specialty milestones and the next accreditation system: an opportunity for the simulation community
.
Simul Healthc
.
2014
;
9
(
3
):
184
191
.
11
Mann
KV.
Theoretical perspectives in medical education: past experiences and future possibilities
.
Med Educ
.
2011
;
45
(
1
):
60
68
.
12
Kessler
D,
Pusic
M,
Chang
TP,
et al.
Impact of just-in-time and just-in-place simulation on intern success with infant lumbar puncture
.
Pediatrics
.
2015
;
135
(
5
):
e1237
e1246
.
13
Kobayashu
L,
Patterson
MD,
Overly
FL,
et al.
Educational and research implications of portable human patient simulation in acute care medicine
.
Acad Emerg Med
.
2008
;
15
(
11
):
1166
1174
.

Author notes

Funding: Dr. Oron's work was partially supported by the National Center for Advancing Translational Sciences of the National Institutes of Health (UL1TR000423).

Competing Interests

Conflict of interest: The authors declare they have no competing interests.