Design, Dissemination, and Assessment of NephSIM: A Mobile-Optimized Nephrology Teaching Tool

The rapid uptake of technology and near ubiquitous use of mobile devices allows for innovative approaches to graduate medical education.¹  Free Open-Access Medical Education (FOAMed) tools have become the preferred modality for learners.^2–7  In the last decade, online learning tools such as blogs, social media, online journal clubs, videos, games, and texting apps have been embraced by the nephrology community and other specialties.^3,6,8–10  Additionally, social media dissemination of educational content enables quick, broad uptake of these materials.

We hypothesized that the Nephrology Simulator (NephSIM; www.nephsim.com), a free, case-based, mobile-optimized approach to nephrology education that capitalizes on iterative feedback, would be widely used and positively received.

This study took place from June 2018 to June 2019. The Health Insurance Portability and Accountability Act–compliant content, created by nephrology fellows and faculty and peer-reviewed by attending nephrologists, is published every 2 to 4 weeks using WordPress. Nephrologists, radiologists, and pathologists provide radiographic, urine microscopy, and pathology images. Approximately 6 nephrology fellows and 4 nephrology attending physicians from 10 institutions contributed content, and the majority of curation and editing was performed by NephSIM cocreators S.S.F. (nephrology attending) and R.H. (nephrology fellow). The majority of content contributors were members of the Nephrology Social Media Collective.

NephSIM is a FOAMed, mobile-optimized tool using interactive cases and real-time iterative feedback to teach nephrology, pathophysiology, and diagnostic approaches. Cases cover topics relevant to various disciplines including internal medicine, nephrology, and pathology. They replicate real-life patient care and are presented as a sequence of labs, imaging, and pathology (figure 1). The user is first presented with a history of present illness (HPI), followed by physical examination, laboratory, and imaging findings. Then, they have the opportunity to form a differential diagnosis from several provided options.

Next, the user considers the utility of additional tests before choosing the highest-yield studies. Feedback is provided based on their selection. The subsequent pathology section includes questions, interactive responses, and labeled images. The user then chooses their final diagnosis. A case summary page provides a referenced description of pathophysiology and how the data funnel together to the final diagnosis.

To date, 34 NephSIM cases have covered glomerular disease, dialysis, transplant, electrolyte and acid-base disorders, and acute kidney injury (provided as online supplemental material). Minor content errors identified during post-publication peer review (eg, e-mail subscribers, social media users) are addressed and corrected by the NephSIM cocreators. NephSIM content is peer reviewed both pre- and post-publication to allow content to be updated quickly secondary to user comments and feedback.

Cases that address certain thematic areas like kidney transplant, palliative care, and vascular access are tagged and identified. In addition, we assign difficulty levels by placing a star next to each case, ranging from 1 to 3 stars (easiest to most difficult). A guide page lists each case and diagnosis, allowing educators to choose specific cases for their teaching.

A national volunteer team consisting of 8 nephrology fellows and 6 attending nephrologists (representing approximately 12 institutions) helped to develop, format, and edit NephSIM cases and material. However, a majority of the cases were completed by the cocreators (S.S.F. and R.H.). Depending on the complexity and the amount of ancillary material that is published, each case takes approximately 3 to 4 hours to develop and complete. In addition, cases are reviewed by content experts when appropriate. Proficiency in WordPress and basic use of Hypertext Markup Language (HTML) were needed to create this teaching tool. No additional tools or resources were required.

NephSIM content was regularly sent to an e-mail list of 445 individuals who subscribed via a sign-up form on the homepage. Content was shared via Twitter (@Neph_SIM) at least 3 times a week by a team of nephrology fellows and attending nephrologists.

Twitter analytics were reviewed, including followers, impressions (number of times a tweet appears to users), and engagement rate (number of times a user interacted with the tweets divided by impressions). Website usage, including the number of views, visitors, and countries accessing the website, was assessed using WordPress analytics. Case completion rates were obtained by calculating the ratio of Diagnosis and Conclusions page views to HPI page views.

Subscribers received an online survey to evaluate their demographics and NephSIM experience (provided as online supplemental material). A 9-item survey (3 close-ended multiple-choice questions, 4 Likert-scale questions with 4 responses, and 1 open-ended question) was sent to all subscribers. The survey was developed by the authors without further testing.

This study was determined to be exempt from Institutional Board Review according to the Mount Sinai Human Research Determination policy.

NephSIM had 100 745 page views between June 2018 and June 2019, representing 17 922 unique visitors from over 100 countries. Thirty-nine percent of views came from the United States, 6.2% from India, 5.9% from the United Kingdom, 5.2% from Mexico, 3.1% from China, and 2.8% from Canada. Sixty-four percent of page views to NephSIM came from mobile devices, 30% from desktop computers, and 6% from tablets. Page views from NephSIM tutorial pages were Pathology 101: 4751 views; Educator's Guide: 3380 views; Acid/Base: 5507 views; and Image Gallery: 4915 views.

To date, 1929 accounts follow NephSIM on Twitter (last accessed June 9, 2019). Over the last 28-day period (May 14–June 10, 2019), tweets from the NephSIM account earned 124 200 impressions with 1500 link clinks and a 3% engagement rate.

The median case completion rate was 69.1% (interquartile range 63.6%–78.0%). Three cases were excluded due to page linking errors.

Seventeen percent (76 of 445) of e-mail subscribers completed the survey. Fifty-one percent (39 of 76) were 31 to 45 years old; 32% (24 of 76) were nephrology fellows, 25% (19 of 76) nephrology attendings, 9% (7 of 76) internal medicine residents, and 1.3% (1 of 76) medical students. Additional participants (33%, 25 of 76) included physician assistants, nursing students, residents in other specialties, and professors and graduate students in other fields.

The majority of participants agreed or somewhat agreed that they use NephSIM for individual learning (68%, 52 of 76) or to teach others (75%, 57 of 76; figure 2). Ninety-six percent (73 of 76) agreed or somewhat agreed that NephSIM was easy to use. Nearly all either agreed or somewhat agreed that they enjoyed using NephSIM (96%, 73 of 76) and planned to continue using it in the future (99%, 75 of 76).

NephSIM, a free, mobile-optimized, online case-based learning tool developed by volunteer nephrology fellows and faculty, demonstrated high use and global reach in its first year of operation. Case completion rate was high, and survey respondents reported high acceptability and intent to continue to use the program. NephSIM was used by fellows, faculty, residents, and other health professionals for teaching and personal learning.

NephSIM's Twitter engagement rate is 3.0%, significantly above the median (0.05%) for all industries (including health and higher education).¹¹  The engagement rate may reflect the popularity of this teaching tool as well as the use of polls, media, and hashtags in the tweets.¹² 

NephSIM delivers content that caters to different learning styles. The Index of Learning Styles describes 4 domains: active versus reflective, sensing versus intuitive, visual versus verbal, and sequential versus global.^13–15  Users of NephSIM, who are provided with infographics and illustrations, actively participate as the case unfolds sequentially, by completing differential diagnoses and receiving both positive and negative iterative feedback. Thus, NephSIM content appeals to a variety of these styles, including dominant ones (active, sensing, visual, and sequential). While NephSIM focused on nephrology cases and targeted nephrologists, users included internal medicine residents, and this design could be replicated by other specialties.

Limitations to these findings include that the survey assessed self-reported learning, rather than externally assessed learning or behavior change. As the survey was developed without testing, questions may have been interpreted differently than intended; recall bias may also have occurred due to delayed timing of the survey. Selection bias is likely as the survey response rate was 17%. Calculation of the case completion rate assumes progression from the first to the final page, although users may have accessed the final page independently. In addition, website and social media analytics that assess NephSIM usage likely overestimated the number of true users of the content.

Our future plans are to study the impact of NephSIM on knowledge acquisition. We aim to incorporate NephSIM into medical school, internal medicine, and nephrology training program curricula with pre-post knowledge assessment. We will continue to diversify content and contributors.

Nephrology fellows and faculty developed and deployed a mobile-optimized, case-based approach to teach nephrology using widely accessible tools, which has grown with global participation and positive assessment.