Background

Paracentesis is a commonly performed bedside procedure that has the potential for serious complications. Therefore, simulation-based education for paracentesis is valuable for clinicians.

Objective

To assess internal medicine residents' procedural skills before and after simulation-based mastery learning on a paracentesis simulator.

Methods

A team with expertise in simulation and procedural skills developed and created a high fidelity, ultrasound-compatible paracentesis simulator. Fifty-eight first-year internal medicine residents completed a mastery learning-based intervention using the paracentesis simulator. Residents underwent baseline skill assessment (pretest) using a 25-item checklist. Residents completed a posttest after a 3-hour education session featuring a demonstration of the procedure, deliberate practice, ultrasound training, and feedback. All residents were expected to meet or exceed a minimum passing score (MPS) at posttest, the key feature of mastery learning. We compared pretest and posttest checklist scores to evaluate the effect of the educational intervention. Residents rated the training sessions.

Results

Residents' paracentesis skills improved from an average pretest score of 33.0% (SD  =  15.2%) to 92.7% (SD  =  5.4%) at posttest (P < .001). After the training intervention, all residents met or exceeded the MPS. The training sessions and realism of the simulation were rated highly by learners.

Conclusion

This study demonstrates the ability of a paracentesis simulator to significantly improve procedural competence.

Editor's Note: The online version of this article contains a physician survey about the paracentesis simulator and the 25-item paracentesis skills checklist.

What was known

Paracentesis is a commonly performed bedside procedure with the potential for complications. Simulation-based procedural training is beneficial for learners.

What is new

Internal medicine residents who participated in a 3-hour session with deliberate practice and feedback improved and all achieved a minimum passing score.

Limitations

Small sample, single site, and single specialty.

Bottom line

Residents′ paracentesis skills improved significantly after a simulation and feedback session that is feasible for most programs.

Internal medicine residents and gastroenterology fellows commonly perform paracentesis procedures. The American Board of Internal Medicine (ABIM) requires residents to be familiar with the indications, complications, and interpretation of this procedure.1 Recently, the Accreditation Council for Graduate Medical Education added paracentesis skills as a 2012 program requirement for gastroenterology fellowship training as well as a requirement that fellows have access to simulation training.2 Despite these recommendations, many procedures are performed by learners who are not appropriately proficient or confident,3,4 and there is variable adherence to the gastroenterology core curriculum regarding hepatology in many programs.5 

Simulation-based mastery learning6 featuring deliberate practice7 gives residents and fellows the opportunity for procedural skills development and feedback. Mastery learning requires that learners meet or exceed a minimum passing score (MPS) on a simulated examination prior to performing the procedure in actual clinical care.6 We previously used simulation-based mastery learning to improve skills of residents and fellows in thoracentesis,8 Advanced Cardiac Life Support,9 and central venous catheter (CVC) insertion.1012 

We hypothesized that the mastery model of procedural training could be successfully extended to paracentesis skills on an ultrasound-compatible simulator that was designed and created at our institution.

Simulator Design and Evaluation

From May to August 2008, various materials and strategies were tested to create an ultrasound-compatible paracentesis model. A prototype was produced and tested to obtain feedback and suggested modifications. Ten identical simulators were pilot tested at a national procedure skills conference held at Northwestern University in October 2009. Twenty-one practicing physicians who participated in the conference tested the simulator and provided feedback about the simulator's realism (provided as online supplemental material, Appendix A).

The paracentesis simulator represents the human abdomen from rib margin to pubis. A foam chassis contains a “cartridge” containing simulated liver, kidneys, pancreas, aorta, intestines, and ascitic fluid. The chassis and cartridge are covered with an ultrasound-compatible silicone-based “skin” which is affixed to the chassis. Realistic ultrasound images can be obtained (figure 1), and a paracentesis can be performed (figure 2). Research and development of the simulator required partial effort of 2 part-time staff to complete. The estimated cost of disposable supplies was $1500 and labor cost $9216. The final cost of materials for each simulator was $57.

FIGURE 1

Ultrasound Image of the Paracentesis Simulator Demonstrating Anatomically Correct Appearance of Internal Organs

FIGURE 1

Ultrasound Image of the Paracentesis Simulator Demonstrating Anatomically Correct Appearance of Internal Organs

Close modal
FIGURE 2

Paracentesis Procedure Using the Simulator

FIGURE 2

Paracentesis Procedure Using the Simulator

Close modal

Use of the Paracentesis Simulator in Resident Education

An educational intervention was performed within the internal medicine residency at Northwestern University from July to September 2010. Participants were all 58 postgraduate year-one (PGY-1) internal medicine residents. The Northwestern University Institutional Review Board approved the study, and all participants provided informed consent.

PGY-1 residents received standardized simulation-based education regarding paracentesis. Two faculty members supervised educational sessions for a total of 5 days. Residents underwent baseline skill assessment (pretest) on the simulator, using a 25-item checklist (provided as online supplemental material, Appendix B). The checklist was developed by one author (J.H.B.) using evidence-based guidelines for paracentesis13,14 and checklist design.812,15,16 The checklist was reviewed for completeness by 2 other authors with expertise and experience in paracentesis, simulation-based education, and checklist design (J.A.V. and D.B.W.).

After undergoing pretesting, all study residents completed a 3-hour educational session featuring a lecture, the New England Journal of Medicine's paracentesis video,14 ultrasound training, and deliberate practice with the paracentesis simulator with directed feedback.7 

Following training, residents were retested (posttest) using the 25-item checklist and were expected to meet or exceed the MPS for simulated paracentesis procedures. Residents who failed to meet the MPS the underwent additional deliberate practice until they were able to meet the MPS. The MPS for the paracentesis clinical skills examination was determined by 9 clinical experts (ABIM-certified physicians, 6 physicians from internal medicine, and 1 physician each from gastroenterology, pulmonary/critical care, and infectious disease) using the Angoff and Hofstee standard setting methods.1719 

Pretest and posttest scores were compared to assess the impact of the educational intervention. Residents responded to a questionnaire about their experience with the simulator and the training sessions after the intervention.

Measurement

Demographic data were obtained from participants including age, sex, United States Medical Licensing Examination (USMLE) Step 1 and 2 scores, and prior clinical experience with paracentesis procedures. All pretests and posttests were graded by a single unblinded instructor (E.R.C.) and were videotaped. A dichotomous scoring scale (done correctly or incorrectly) was used for each item.

A faculty instructor with expertise in scoring clinical skills examinations, who was blinded to pretest and posttest status of each examinee (J.H.B.), rescored a 50% random sample of the videotaped examinations to assess interrater reliability. Course evaluation questionnaires have been previously published,8,9,12 and answers were recorded on a 5-point Likert scale (where 1  =  strongly disagree, and 5  =  strongly agree).

Statistical Analysis

The Cohen kappa coefficient was used to assess interrater reliability. Pretest and posttest checklist scores were compared using a paired t-test. Multiple linear regression was used to assess the association of posttest score with pretest performance, age, sex, USMLE score, and clinical experience. Analyses were performed using PASW version 18.0 statistical software (SPSS, Inc., Chicago, IL).

All 58 PGY-1 residents consented to participate and completed the entire protocol (table 1).

TABLE 1

Demographic Data of PGY1 Residents Undergoing Paracentesis Simulation Training

Demographic Data of PGY1 Residents Undergoing Paracentesis Simulation Training
Demographic Data of PGY1 Residents Undergoing Paracentesis Simulation Training

The MPS for the skills checklist was set at 83%. Interobserver agreement for the educational intervention skills checklist was high (κn  =  .87). Paracentesis checklist skills improved from an average pretest score of 8.25/25 (33.0%; SD  =  15.2%) to 23.2/25 (92.7%; SD  =  5.4%) at posttest (P < .001) after the training intervention (figure 3).

FIGURE 3

Pretest and Posttest Paracentesis Skill Scores on the 25-Item Checklist SEM  =  standard error of measurement; MPS  =  minimum passing score

FIGURE 3

Pretest and Posttest Paracentesis Skill Scores on the 25-Item Checklist SEM  =  standard error of measurement; MPS  =  minimum passing score

Close modal

Two residents (3%) failed to meet the MPS at posttest. They subsequently underwent additional deliberate practice and were retested. Both residents reached the MPS within 1 hour of further practice.

Regression analysis showed no associations between posttest scores and pretest performance, age, sex, USMLE Step 1 and 2 scores, or prior clinical experience with paracentesis procedures. Residents rated the training sessions highly (table 2).

TABLE 2

Residents' Post-training Questionnaire Results

Residents' Post-training Questionnaire Results
Residents' Post-training Questionnaire Results

To our knowledge, this is the first study to use the mastery learning model for simulation-based paracentesis education. Our data demonstrate significant improvement in paracentesis skills of PGY-1 residents after simulation-based education. The magnitude of skill improvement using the paracentesis simulator was similar to that shown in earlier studies with commercial-grade simulators for thoracentesis,8 Advanced Cardiac Life Support,9,15 and CVC insertion.1012 These interventions have also been linked to improved patient care quality.10,11,20,21 Our results also support the policy of the ABIM1 which states that simulation should be used to verify trainees′ skill before performing invasive procedures in actual patient care.

This study contributes to what is known about paracentesis procedural skill acquisition among medical learners. When assessed rigorously, there was no association among demographic variables such as age, sex, or USMLE examination scores, and paracentesis clinical skills. Additionally, our study found no relationship between the number of paracentesis procedures performed in actual clinical care and measured procedural skills. However, a difference may have been unlikely, as most residents had previously performed 0–1 procedures. Past research on thoracentesis,8 CVC insertion,10,11 and endoscopic balloon dilation of the esophagus22 also shows no association between the number of procedures performed and procedural skill. This is problematic because a 2003 Core Curriculum developed by the Gastroenterology Leadership Council for gastroenterology fellows recommends that fellows demonstrate competence in paracentesis by performing at least 20 procedures.23 Clinical experience is not a proxy for skill,24 and our findings are a reminder that competence should not be assumed after a set number of procedures.

This study has several limitations. The tools used to create a “homemade” simulator and the faculty commitment required for simulation training may not be available at other institutions. Training programs should consider the benefits of simulation as well as new training requirements when planning educational interventions. The degree of improvement in posttest scores recorded in our study may be reduced because our paracentesis model was used for both education and testing and we did not use a true “control” group. However, this should not diminish the pronounced impact simulation-based training had on paracentesis skills of first-year residents. Additionally, post-testing was performed immediately after training, thereby enhancing recall of procedural steps. In the service of translational science goals,25 we have not yet linked improved paracentesis skills in the simulated environment to improved health care delivery (higher procedural success rates, lower referral rate to interventional radiology) or better patient outcomes (lower complication rates).

This study demonstrates the ability of simulation-based education to improve paracentesis skills in internal medicine residents. Use of simulation as an adjunct to clinical training helps students meet residency and fellowship accreditation requirements and is feasible and well accepted by learners. Further study should address the effect of training on the long-term retention of paracentesis skills and health care outcomes.

1
American Board of Internal Medicine
.
Internal medicine policies: procedures required for internal medicine
. .
2
Accreditation Council for Graduate Medical Education
.
ACGME program requirements for graduate medical in gastroenterology
. .
3
Wigton
RS
,
Blank
LL
,
Nicolas
JA
,
Tape
TG
.
Procedural skills training in internal medicine residencies: a survey of program directors
.
Ann Intern Med
.
1989
;
111
(
11
):
932
938
.
4
Huang
GC
,
Smith
CC
,
Gordon
CE
,
Feller-Kopman
DJ
,
Davis
RB
,
Phillips
RS
.
Beyond the comfort zone: residents assess their comfort performing inpatient medicine procedures
.
Am J Med
.
2006
;
119
(
1
):
71.e17
e24
.
5
Guardino
JM
,
Proctor
DD
,
Lopez
R
,
Carey
W
.
Utilization of and adherence to the gastroenterology core curriculum on hepatology training during a gastrointestinal fellowship
.
Clin Gastroenterol Hepatol
.
2008
;
6
(
6
):
682
688
.
6
Block
JH
,
ed
.
Mastery Learning: Theory and Practice
.
New York, NY
:
Holt, Rinehart, and Winston
;
1971
.
7
Ericsson
KA
.
Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains
.
Acad Med
.
2004
;
79
(
suppl 10
):
S70
S81
.
8
Wayne
DB
,
Barsuk
JH
,
O'Leary
KJ
,
Fudala
MJ
,
McGaghie
WC
.
Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice
.
J Hosp Med
.
2008
;
3
(
1
):
48
54
.
9
Wayne
DB
,
Butter
J
,
Siddall
VJ
,
Fudala
MJ
,
Wade
LD
,
Feinglass
J
.
Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice
.
J Gen Intern Med
.
2006
;
21
(
3
):
251
256
.
10
Barsuk
JH
,
McGaghie
WC
,
Cohen
ER
,
Balachandran
JS
,
Wayne
DB
.
Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit
.
J Hosp Med
.
2009
;
4
(
7
):
397
403
.
11
Barsuk
JH
,
McGaghie
WC
,
Cohen
ER
,
O'Leary
KJ
,
Wayne
DB
.
Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit
.
Crit Care Med
.
2009
;
37
(
10
):
2697
2701
.
12
Barsuk
JH
,
Ahya
SN
,
Cohen
ER
,
McGaghie
WC
,
Wayne
DB
.
Mastery learning of temporary hemodialysis catheter insertion by nephrology fellows using simulation technology and deliberate practice
.
Am J Kidney Dis
.
2009
;
54
(
1
):
70
76
.
13
Runyon
BA
.
Management of adult patients with ascites due to cirrhosis: an update
.
Hepatology
.
2009
;
49
(
6
):
2087
2107
.
14
Thomsen
TW
,
Shaffer
RW
,
White
B
,
Setnik
GS
.
Videos in clinical medicine. Paracentesis
.
N Engl J Med
.
2006
;
355
(
19
):
e21
.
15
Wayne
DB
,
Butter
J
,
Siddall
VJ
,
Fudala
MJ
,
Linquist
LA
,
Feinglass
J
,
et al.
Simulation-based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial
.
Teach Learn Med
.
2005
;
17
(
3
):
210
216
.
16
Stufflebeam
D
.
Guidelines for developing evaluation checklists: the checklists development checklist (CDC)
. .
17
Cusimano
MD
.
Standard setting in medical education
.
Acad Med
.
1996
;
71
(
suppl 10
):
S112
S120
.
18
Wayne
DB
,
Fudala
MJ
,
Butter
J
,
Siddall
VJ
,
Feinglass
J
,
Wade
LD
.
Comparison of two standard-setting methods for advanced cardiac life support training
.
Acad Med
.
2005
;
80
(
suppl 10
):
S63
S66
.
19
Norcini
J
,
Guille
R
.
Combining tests and setting standards
.
In:
Norman
GR
,
Van der Vleuten
CPM
,
Newble
DI
,
eds
.
International Handbook of Research in Medical Education
.
Dordrecht, The Netherlands
:
Kluwer Academic Publishers
;
2002
:
811
.
20
Wayne
DB
,
Didwania
A
,
Feinglass
J
,
Fudala
MJ
,
Barsuk
JH
,
McGaghie
WC
.
Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study
.
Chest
.
2008
;
133
(
1
):
56
61
.
21
Barsuk
JH
,
Cohen
ER
,
Feinglass
J
,
McGaghie
WC
,
Wayne
DB
.
Use of simulation-based education to reduce catheter-related bloodstream infections
.
Arch Intern Med
.
2009
;
169
(
15
):
1420
1423
.
22
Kim
H
,
Pandolfino
JE
,
Komanduri
S
,
Hirano
I
,
Cohen
ER
,
Wayne
DB
.
Use of a continuing medical education course to improve fellows' knowledge and skills in esophageal disorders
.
Dis Esophagus
.
2011
;
24
(
6
):
388
394
.
23
Training the gastroenterologist of the future: the Gastroenterology Core Curriculum
.
Gastroenterology
.
2003
;
124
(
4
):
1055
1104
.
24
Choudhry
NK
,
Fletcher
RH
,
Soumerai
SB
.
Systematic review: the relationship between clinical experience and quality of health care
.
Ann Intern Med
.
2005
;
142
(
4
):
260
273
.
25
McGaghie
WC
,
Draycott
TJ
,
Dunn
WF
,
Lopez
CM
,
Stefanidis
D
.
Evaluating the impact of simulation on translational patient outcomes
.
Simul Healthc
.
2011
;
6
(
suppl
):
S42
S47
.

Author notes

All authors are at Northwestern University Feinberg School of Medicine. Jeffrey H. Barsuk, MD, is Associate Professor of Medicine; Elaine R. Cohen, BA, is Research Assistant; John A. Vozenilek, MD, is Director of the Center for Simulation Technology and Immersive Learning and Associate Professor of Emergency Medicine; Lanty M. O'Connor, BA, is Manager of Simulation Technologies in the Center for Simulation Technology and Immersive Learning; William C. McGaghie, PhD, is Jacob R. Suker, MD, Professor of Medical Education; and Diane B. Wayne, MD, is Vice Chair of Education, Internal Medicine Residency Program Director, and Professor of Medicine.

Funding: The authors report no external funding source for this study.

The authors would like to acknowledge the efforts of several individuals employed by the Simulation Technology and Immersive Learning program at Northwestern University who contributed to building the paracentesis simulator: Gregory Podolej, Patrick Flannery, Mary O'Donnell, Caitlin MacGregor, Robert Metzler, Lauren Davis, Jennifer Haag, and Suha Hassen. We thank the Northwestern University internal medicine residents for their dedication to education and patient care. We acknowledge Drs. Douglas Vaughan, and Raymond Curry for their support and encouragement of this work. Please contact Dr. John Vozenilek for information on obtaining the paracentesis simulator ([email protected]).

Supplementary data