Comparison of Personal Video Technology for Teaching and Assessment of Surgical Skills

Video recording and playback for feedback, coaching, and assessment of surgical skills has increased dramatically in recent years.^1,2  Video use is supported by educational theory³  and predicated on equipment miniaturization as found in minimally invasive surgery. Video review allows physicians to observe aspects of their performance rather than relying on memory alone. Identified issues can be addressed on subsequent attempts, as advocated in a deliberate practice model of skill acquisition.⁴  Video review has been shown to improve surgical skills in multiple studies^1,5–7  and has also been used for assessment.^1,6,7  Procedural videos are increasingly being used formally in structured demonstration videos and informally on social media for independent learning (eg, YouTube).⁸ 

Video recordings of open surgical skills have languished behind those of minimally invasive techniques due to the need for additional video equipment and the difficulty of accessing the area of interest (AOI) within the sterile field. Cameras built into operating room infrastructure (such as overhead lights)^9,10  have attempted to bridge this gap, but still suffer from obstructed views. Improvements in personal electronic video technology have opened new avenues for recording open surgical skills.

The purpose of this study was to compare 5 personal device technologies on their utility for capturing video of open surgical skills, and to determine advantages and disadvantages of each type of device in various research settings.

Videos of medical students' open surgical skills that were used for feedback and assessment purposes in another study⁷  were analyzed for utility. These videos were of first- and second-year medical students who are interested in surgery performing a benchtop small bowel hand-sewn anastomosis or skin lesion excision and closure. Students required between 20 and 75 minutes to complete 1 practice attempt. Each student mounted their own video-recording device—smartphone/tablet (MOB) or laptop (LAP)—on the benchtop to ensure their identity remained hidden while the operative field was visible (figure 1). Both audio and video footage were recorded. MOB were mounted using improvised stands of examination glove boxes and bulldog clips, or alternatively using head mounts made of rubber bands and bulldog clips (figure 2).

To improve device selection for this study a tripod-mounted Canon 5D Mark III with 24-104 mm lens (DSLR), a GoPro Hero 5 (SC) tripod and head mounted, and a 1080p spy-camera pair of glasses with the lens mounted into the arm (SPY) were also employed. These additional devices were used by students to record extra footage for the purpose of this device utility study. All devices were set to fully automatic recording, with image stabilization where available. Students' devices were of varying ages from major manufacturers. Videos were transferred using file-sharing or direct hardware transfer onto external hard drive storage via a standardized process. Videos were analyzed by 2 investigators using default or basic video software (eg, VLC, Quicktime).

The Queen's University General Research Ethics Board approved the recording of medical students.

For the purpose of this article, utility is defined as image quality, hardware, mounting options, and availability of the technology. Each device was assessed for its utility of capturing open surgical skills. Overall utility score was calculated by averaging multiple elements within each domain (table). Two investigators trained in observation-based technical assessment rated each device after reviewing all video footage using a 5-point Likert scale (1, poor; 3, functional; 5, excellent). Consensus among investigators was achieved by discussion. Audio footage was muted and only video footage was examined to improve participant blinding. Investigators were also blinded to the specific device used to record each video.

The 5 devices were assessed for their utility in capturing open surgical skills (table). Each device recorded at least 1 hour of footage, with videos lasting between 15 and 110 minutes long. Investigators examined a total of 150 hours of video, the majority of which came from LAP or MOB devices (approximately 70 hours each).

The highest utility devices (4.25) were DSLR and SC, with LAP and MOB rating very good (4).

The 5 personal device technologies (MOB, LAP, SC, DSLR, SPY) successfully recorded open surgical skills in a simulated setting for feedback and assessment purposes. Each device has its own niche (table). However, based on our experiences, we recommend the following:

• Need for high-quality imaging: Digital SLR and sports cameras provide the highest-quality video images. DSLR provide zoomed videos from an offset position, while sports cameras provide wider-angle images closer to the AOI.

• Lengthy tasks: Laptops are best at recording long tasks, due to increased storage and battery life, especially with sequential recordings. This was only significant for us when tasks took longer than 45 minutes.

• Multiple concurrent sessions (eg, large sample sizes): The ubiquitous nature of laptops and smartphones allows rapid access to multiple devices, without additional financial burden. The authors used this capability to facilitate concurrent recording of 15 procedures, with participants using their own devices.

• Difficult access to AOI: It is difficult to video record procedures when the AOI is obstructed by assistants, or deep within a body cavity. Novel first-person views available with sports cameras and spyglasses allow the camera to be brought closer to the AOI without compromising sterility. Sports cameras with built-in image stabilization provide higher-quality video than spyglasses.

• On-demand recording of procedures: Smartphones and laptops are ubiquitous in our daily lives and are often available devices for on-demand video capture.

Our results highlight that video capture of open surgical skills requires deliberate planning to identify an appropriate device and location from which to capture the video. However, as described previously, researchers and educators have a variety of small, high-utility devices within their armamentarium that allows video to be captured in a manner similar to minimally invasive procedures (eg, laparoscopy or robotic).

Our study has limitations. We used a consensus approach to determine device utility, rather than quantitatively analyzing independent assessments to determine the overall utility for 3 reasons: (1) investigators were blinded to the type of device used by the student, although it was often possible to distinguish between laptop and smartphone video; (2) video quality within a device category depended on the age of the device, with old devices providing poorer quality than new devices, making it difficult to provide an overall score without potential recall bias; and (3) the investigators had differing exposure to the non-video aspects of device utility (eg, file transfer and storage capacity). This variability would have affected interclass correlation, whereas a consensus method allowed these aspects to be identified and discussed in a more comprehensive manner. This study was conducted within a benchtop simulation laboratory with medical students performing simulated tasks, so it is difficult to establish how the results would be extrapolated beyond this setting, limiting generalizability.

Future research should focus on translating this study into the operating room environment, as well as examining staff and patient perceptions to the use of video-recording devices in the operating room.

Video recording open simulated procedures using personal devices is not only feasible but also can provide strong evidence for assessment and feedback purposes. Sports cameras and DSLR provide the greatest overall utility while smartphones and laptops have the greatest flexibility.