ABSTRACT

Background

Digital innovations have the potential to enhance current graduate medical education strategies.

Objective

We assessed the scope, reach, and effectiveness of Nephrology Simulator (NephSIM), a free, mobile-optimized, nephrology educational tool designed to teach pathophysiology with a diagnostic approach using interactive cases.

Methods

NephSIM, launched in June 2018, was designed as a mobile-optimized website with peer-reviewed content in WordPress. Content, including interactive cases with iterative feedback, infographics, and tutorials, was developed by nephrology fellows and attending nephrologists. The teaching tool was shared via an e-mail subscriber list and Twitter. Website usage data and Twitter analytics were reviewed. Case content was categorized, and the case completion rate was calculated. A self-report survey was sent to subscribers to assess their demographics and experience.

Results

Thirty-four cases have been published to date and represent a variety of nephrology topics. There have been 100 745 page views between June 2018 and June 2019, representing 17 922 unique visitors from more than 100 countries. There are 1929 accounts that follow NephSIM on Twitter. Tweets received 124 200 impressions and a 3% engagement rate. Median case completion rate was 69% (interquartile range 64%–78%). Our survey response rate was 17% (76 of 445). Nearly all NephSIM users rated the platform highly in terms of satisfaction and usability, and planned to continue using in the future.

Conclusions

The development of a mobile-optimized, case-based teaching approach by nephrology fellows and faculty is feasible and has demonstrated global participation and high levels of learner satisfaction.

Introduction

The rapid uptake of technology and near ubiquitous use of mobile devices allows for innovative approaches to graduate medical education.1  Free Open-Access Medical Education (FOAMed) tools have become the preferred modality for learners.27  In the last decade, online learning tools such as blogs, social media, online journal clubs, videos, games, and texting apps have been embraced by the nephrology community and other specialties.3,6,810  Additionally, social media dissemination of educational content enables quick, broad uptake of these materials.

We hypothesized that the Nephrology Simulator (NephSIM; www.nephsim.com), a free, case-based, mobile-optimized approach to nephrology education that capitalizes on iterative feedback, would be widely used and positively received.

Methods

Settings and Participants

This study took place from June 2018 to June 2019. The Health Insurance Portability and Accountability Act–compliant content, created by nephrology fellows and faculty and peer-reviewed by attending nephrologists, is published every 2 to 4 weeks using WordPress. Nephrologists, radiologists, and pathologists provide radiographic, urine microscopy, and pathology images. Approximately 6 nephrology fellows and 4 nephrology attending physicians from 10 institutions contributed content, and the majority of curation and editing was performed by NephSIM cocreators S.S.F. (nephrology attending) and R.H. (nephrology fellow). The majority of content contributors were members of the Nephrology Social Media Collective.

Intervention

NephSIM is a FOAMed, mobile-optimized tool using interactive cases and real-time iterative feedback to teach nephrology, pathophysiology, and diagnostic approaches. Cases cover topics relevant to various disciplines including internal medicine, nephrology, and pathology. They replicate real-life patient care and are presented as a sequence of labs, imaging, and pathology (figure 1). The user is first presented with a history of present illness (HPI), followed by physical examination, laboratory, and imaging findings. Then, they have the opportunity to form a differential diagnosis from several provided options.

figure 1

NephSIM Case Journey

Abbreviations: HPI, history of present illness; CMP, comprehensive metabolic panel; CBC, complete blood count; PT, prothrombin time; PTT, partial thromboplastin time; INR, international normalized ratio; UA, urinalysis.

figure 1

NephSIM Case Journey

Abbreviations: HPI, history of present illness; CMP, comprehensive metabolic panel; CBC, complete blood count; PT, prothrombin time; PTT, partial thromboplastin time; INR, international normalized ratio; UA, urinalysis.

Next, the user considers the utility of additional tests before choosing the highest-yield studies. Feedback is provided based on their selection. The subsequent pathology section includes questions, interactive responses, and labeled images. The user then chooses their final diagnosis. A case summary page provides a referenced description of pathophysiology and how the data funnel together to the final diagnosis.

To date, 34 NephSIM cases have covered glomerular disease, dialysis, transplant, electrolyte and acid-base disorders, and acute kidney injury (provided as online supplemental material). Minor content errors identified during post-publication peer review (eg, e-mail subscribers, social media users) are addressed and corrected by the NephSIM cocreators. NephSIM content is peer reviewed both pre- and post-publication to allow content to be updated quickly secondary to user comments and feedback.

Cases that address certain thematic areas like kidney transplant, palliative care, and vascular access are tagged and identified. In addition, we assign difficulty levels by placing a star next to each case, ranging from 1 to 3 stars (easiest to most difficult). A guide page lists each case and diagnosis, allowing educators to choose specific cases for their teaching.

A national volunteer team consisting of 8 nephrology fellows and 6 attending nephrologists (representing approximately 12 institutions) helped to develop, format, and edit NephSIM cases and material. However, a majority of the cases were completed by the cocreators (S.S.F. and R.H.). Depending on the complexity and the amount of ancillary material that is published, each case takes approximately 3 to 4 hours to develop and complete. In addition, cases are reviewed by content experts when appropriate. Proficiency in WordPress and basic use of Hypertext Markup Language (HTML) were needed to create this teaching tool. No additional tools or resources were required.

NephSIM content was regularly sent to an e-mail list of 445 individuals who subscribed via a sign-up form on the homepage. Content was shared via Twitter (@Neph_SIM) at least 3 times a week by a team of nephrology fellows and attending nephrologists.

Outcomes

Twitter analytics were reviewed, including followers, impressions (number of times a tweet appears to users), and engagement rate (number of times a user interacted with the tweets divided by impressions). Website usage, including the number of views, visitors, and countries accessing the website, was assessed using WordPress analytics. Case completion rates were obtained by calculating the ratio of Diagnosis and Conclusions page views to HPI page views.

Subscribers received an online survey to evaluate their demographics and NephSIM experience (provided as online supplemental material). A 9-item survey (3 close-ended multiple-choice questions, 4 Likert-scale questions with 4 responses, and 1 open-ended question) was sent to all subscribers. The survey was developed by the authors without further testing.

This study was determined to be exempt from Institutional Board Review according to the Mount Sinai Human Research Determination policy.

Results

Website Usage and Twitter Analytics

NephSIM had 100 745 page views between June 2018 and June 2019, representing 17 922 unique visitors from over 100 countries. Thirty-nine percent of views came from the United States, 6.2% from India, 5.9% from the United Kingdom, 5.2% from Mexico, 3.1% from China, and 2.8% from Canada. Sixty-four percent of page views to NephSIM came from mobile devices, 30% from desktop computers, and 6% from tablets. Page views from NephSIM tutorial pages were Pathology 101: 4751 views; Educator's Guide: 3380 views; Acid/Base: 5507 views; and Image Gallery: 4915 views.

To date, 1929 accounts follow NephSIM on Twitter (last accessed June 9, 2019). Over the last 28-day period (May 14–June 10, 2019), tweets from the NephSIM account earned 124 200 impressions with 1500 link clinks and a 3% engagement rate.

The median case completion rate was 69.1% (interquartile range 63.6%–78.0%). Three cases were excluded due to page linking errors.

Survey Results

Seventeen percent (76 of 445) of e-mail subscribers completed the survey. Fifty-one percent (39 of 76) were 31 to 45 years old; 32% (24 of 76) were nephrology fellows, 25% (19 of 76) nephrology attendings, 9% (7 of 76) internal medicine residents, and 1.3% (1 of 76) medical students. Additional participants (33%, 25 of 76) included physician assistants, nursing students, residents in other specialties, and professors and graduate students in other fields.

The majority of participants agreed or somewhat agreed that they use NephSIM for individual learning (68%, 52 of 76) or to teach others (75%, 57 of 76; figure 2). Ninety-six percent (73 of 76) agreed or somewhat agreed that NephSIM was easy to use. Nearly all either agreed or somewhat agreed that they enjoyed using NephSIM (96%, 73 of 76) and planned to continue using it in the future (99%, 75 of 76).

figure 2

NephSIM Subscriber Responses to Self-Report Survey (n = 77)

figure 2

NephSIM Subscriber Responses to Self-Report Survey (n = 77)

Discussion

NephSIM, a free, mobile-optimized, online case-based learning tool developed by volunteer nephrology fellows and faculty, demonstrated high use and global reach in its first year of operation. Case completion rate was high, and survey respondents reported high acceptability and intent to continue to use the program. NephSIM was used by fellows, faculty, residents, and other health professionals for teaching and personal learning.

NephSIM's Twitter engagement rate is 3.0%, significantly above the median (0.05%) for all industries (including health and higher education).11  The engagement rate may reflect the popularity of this teaching tool as well as the use of polls, media, and hashtags in the tweets.12 

NephSIM delivers content that caters to different learning styles. The Index of Learning Styles describes 4 domains: active versus reflective, sensing versus intuitive, visual versus verbal, and sequential versus global.1315  Users of NephSIM, who are provided with infographics and illustrations, actively participate as the case unfolds sequentially, by completing differential diagnoses and receiving both positive and negative iterative feedback. Thus, NephSIM content appeals to a variety of these styles, including dominant ones (active, sensing, visual, and sequential). While NephSIM focused on nephrology cases and targeted nephrologists, users included internal medicine residents, and this design could be replicated by other specialties.

Limitations to these findings include that the survey assessed self-reported learning, rather than externally assessed learning or behavior change. As the survey was developed without testing, questions may have been interpreted differently than intended; recall bias may also have occurred due to delayed timing of the survey. Selection bias is likely as the survey response rate was 17%. Calculation of the case completion rate assumes progression from the first to the final page, although users may have accessed the final page independently. In addition, website and social media analytics that assess NephSIM usage likely overestimated the number of true users of the content.

Our future plans are to study the impact of NephSIM on knowledge acquisition. We aim to incorporate NephSIM into medical school, internal medicine, and nephrology training program curricula with pre-post knowledge assessment. We will continue to diversify content and contributors.

Conclusions

Nephrology fellows and faculty developed and deployed a mobile-optimized, case-based approach to teach nephrology using widely accessible tools, which has grown with global participation and positive assessment.

References

References
1
Pew Research Center
.
Social Media Fact Sheet
. ,
2019
.
2
Mallin
M,
Schlein
S,
Doctor
S,
Stroud
S,
Dawson
M,
Fix
M.
A survey of the current utilization of asynchronous education among emergency medicine residents in the United States
.
Acad Med
.
2014
;
89
(
4
):
598
601
. doi:.
3
Rope
RW,
Pivert
KA,
Parker
MG,
Sozio
SM,
Merell
SB.
Education in nephrology fellowship: a survey-based needs assessment
.
J Am Soc Nephrol
.
2017
;
28
(
7
):
1983
1990
. doi:.
4
Purdy
E,
Thoma
B,
Bednarczyk
J,
Migneault
D,
Sherbino
J.
The use of free online educational resources by Canadian emergency medicine residents and program directors
.
CJEM
.
2015
;
17
(
2
):
101
106
. doi:.
5
Masters
K,
Ellaway
RH,
Topps
D,
Archibald
D,
Hogue
RJ.
Mobile technologies in medical education: AMEE Guide No. 105
.
Med Teach
.
2016
;
38
(
6
):
537
549
. doi:.
6
Colbert
GB,
Topf
J,
Jhaveri
KD,
Oates
T,
Rheault
MN,
Shah
S,
et al.
The social media revolution in nephrology education
.
Kidney Int Rep
.
2018
;
3
(
3
):
519
529
. doi:.
7
Carley
S,
Beardsell
I,
May
N,
Crowe
L,
Baombe
J,
Grayson
A,
et al.
Social-media-enabled learning in emergency medicine: a case study of the growth, engagement and impact of a free open access medical education blog
.
Postgrad Med J
.
2018
;
94
(
1108
):
92
96
. doi:.
8
Sparks
MA,
O'Seaghdha
CM,
Sethi
SK,
Jhaveri
KD.
Embracing the internet as a means of enhancing medical education in nephrology
.
Am J Kidney Dis
.
2011
;
58
(
4
):
512
518
. doi:.
9
Poyan Mehr
A,
Sadeghi-Najafabadi
M,
Chau
K,
Messmer
J,
Pai
R,
Roy
N,
et al.
The Glomerular Disease Study and Trial consortium: a grassroots initiative to foster collaboration and innovation
.
Kidney Int Rep
.
2019
;
4
(
1
):
20
29
. doi:.
10
Jhaveri
KD,
Pascarelli
B,
Hasan
A,
Kozikowski
A,
Fishbane
S,
Pekmezaris
R.
‘WhatsApp®'ening in nephrology training
[published online April 30,
2019]
. Clin Kidney J. doi:.
11
Rival
IQ.
2019 Social Media Industry Benchmark Report
. ,
2019
.
12
Wadhwa
V,
Latimer
E,
Chatterjee
K,
McCarty
J,
Fitzgerald
RT.
Maximizing the tweet engagement rate in academia: analysis of the AJNR Twitter feed
.
Am J Neuroradiol
.
2017
;
38
(
10
):
1866
1868
. doi:.
13
Quinn
MM,
Smith
T,
Kalmar
EL,
Burgoon
JM.
What type of learner are your students? Preferred learning styles of undergraduate gross anatomy students according to the index of learning styles questionnaire
.
Anat Sci Educ
.
2018
;
11
(
4
):
358
365
. doi:.
14
Felder
R.
Learning and teaching styles in engineering education
.
J Eng Educ
.
1988
;
78
:
674
681
.
15
Felder
RM,
Spurlin
JE.
Applications, reliability and validity of the index of learning styles
.
Int J Eng Educ
.
2005
;
21
:
103
112
.

Author notes

Editor's Note: The online version of this article contains a list of NephSIM cases and the survey used in the study.

Funding: This work was supported by the American Society of Nephrology Innovations in Kidney Education Contest and NephJC, a nonprofit organization that supports the work of NephJC and the Nephrology Social Media Collective.

Competing Interests

Conflict of interest: The authors declare they have no competing interests.

The authors would like to thank Promiti Islam for her critical review of the manuscript and to NephJC and the Nephrology Social Media Collective for supporting this project.

Supplementary data