A comparison of virtual and in-person instruction in a physical examination course during the COVID-19 pandemic

The unexpected nature of the COVID-19 pandemic disrupted the regular operations of professional programs. Institutes of higher education were driven to adjust their curriculum to be taught online, for social distancing, in a relatively short amount of time. Instructors have had to prepare and deliver their classes from home, with all the practical and technical challenges this entails, and often without proper technical support.¹  A significant challenge for university instructors has been their lack of the pedagogical content knowledge²  needed for teaching online.^3,4  Online instruction has already greatly expanded in the past 2 decades even prior to the pandemic. Allen and colleagues found that the percentage of students enrolled in at least 1 online course in American degree-granting colleges and universities increased over 3-fold from 9.6% in fall 2002 to 32.0% in fall 2011.⁵  Instructors are not uniform in their perception of web-based platforms. Some view it as an alternative method for presenting the traditional content, whereas others seek out innovative ways of using such platforms to improve student engagement and thus their learning outcomes.^6,7  Online education has evolved to include a diverse array of tools, resources, pedagogical approaches, organizational arrangements, and forms of interaction, monitoring, and support with many possible combinations of substitution and integration.^8–10  Today, online learning is a part of student experiences for a sizable number of university students in many countries.^7,11,12  Citing existing literature, Cao and Sakchutchawan¹³  found that female, older, working, and part-time students, as well as those with family obligations, are more likely to gravitate toward online learning compared with their counterparts in school. Evidence suggests that students experience different patterns of engagement between online and traditional face-to-face courses.¹⁴  Otter et al¹¹  noted that students in online-only classes felt more disconnected from their peers and instructors, more obliged to be self-directed in their studies, and less aided by their instructors. Although social connectedness can be derived online,¹⁵  most students believe that face-to-face contact is essential for building a sense of community.¹⁶  Yet, other studies have suggested that the less confrontational or personal nature of online learning might encourage introverted students to engage more and feel less social pressure.^17,18 

Previous research studies explored the impact of learning environments in relation to learning outcomes. These studies have produced conflicting results. A study conducted by Thirunarayanan and Perez-Prad¹⁹  showed that there was no statistically significant posttest difference between the online and campus learning groups. McLaren²⁰  did a similar comparison in an undergraduate business statistics course and found successful completion of the course to be independent of the mode of instruction. Moazami et al²¹  examined the effects of virtual versus traditional education in endodontics dental students and concluded that virtual learning was more effective than traditional learning. Hugenholtz et al²²  demonstrated that both virtual and traditional learning methods for continuing medical education effectively enhanced the knowledge of physicians with no significant difference. Finally, a review study on articles regarding continuing medical education via virtual and traditional learning published from 1996 to 2004 showed that virtual learning was as effective as traditional learning in most studies, but 6 studies reported superior efficacy of virtual learning.²³ 

Physical examination (PE) skills are essential to the practice of clinical care. Traditionally, students study and practice their PE skills in an in-person setting on campus because PE skills involve considerable time spent in hands-on learning. Some instructors have chosen to incorporate online technologies into their PE courses. Mir et al used PE video demonstration to replace live demonstration and found that it resulted in no difference in student performance on a 5-station objective structured clinical examination (OSCE) evaluating the performance of knee, abdomen, “motor function,” thyroid, and pulse examination skills.²⁴  Dinh et al²⁵  evaluated the effects of adding point-of-care ultrasound to PE sessions and found that its use led to an increased number of students who received outstanding scores on an OSCE.

However, virtual PE classes remain uncommon. Our search for virtual PE classes in chiropractic education programs yielded no results. Owing to the COVID-19 pandemic, our school began to teach PE classes in a virtual setting including the lecture and lab portions. This was a brand-new experience for our instructors. We hypothesized that the traditional in-person PE lab learning model would be more effective than the virtual lab-learning model; therefore this retrospective study aimed to compare the effectiveness and student academic performance of PE class with online lab vs in-person lab.

This is a retrospective study. Preexisting assessment data of 69 students enrolled in the Head and Neck PE course (July 2020 to September 2020) were analyzed for this study. This course was structured in the 3rd quarter of a 13-quarter program. This research was deemed exempt from the need for full review by the Palmer College of Chiropractic institutional review board.

The pandemic protocol comprised three 50-minute labs and one 50-minute lecture each week. Students completed all lectures online but had the option to attend the lab in person or virtually. The in-person labs were broadcast simultaneously online as virtual labs. Relevant class materials, including PowerPoint (Microsoft Corp, Redmond, WA, USA) slides and videos, were available to all students on the teaching platform (D2L Corp, Kitchener, ON, Canada). Data for after-school practice and learning were self-reported.

Students' academic performances were evaluated using 8 weekly quizzes, 2 formative written examinations, and 2 OSCEs. The weekly quizzes and written examinations were administered online with a single best-response answer in a multiple-choice format. The weekly quizzes covered the materials that students learned from lab classes, while the written exams tested knowledge that they learned from both lab classes and lectures. Last, the OSCE was scored by 2 instructors. The course director first graded each examinee in person followed by a second instructor who graded the students using video playback. The students' final scores were the averages of both instructors' scores.

Data were summarized and analyzed using SPSS Version 22 (IBM Corp, Armonk, NY, USA). Statistical test assumptions were verified and p values less than .05 were considered significant. An independent t test was applied to compare the scores acquired from students in 2 groups (virtual vs in-person). Pearson correlation test was used to evaluate the relationship between in-person and online class attendance test scores.

This sample of convenience included 31 females and 38 males, with ages ranging from 21 to 40 years (25.1 ± 3.7 years, mean ± SD).

Pearson correlation showed that OSCE and weekly quiz scores were positively correlated with in-person class attendance (p = .000, r = .619 and p = .000, r = .488, respectively). According to in-person class attendance, participants were broken down into 2 groups: (1) higher than 50% attendance rates, and (2) 50% or lower attendance rates. Results of the independent samples t test indicated that the mean OSCE (p = .000) and quiz scores (p = .001) for group 1 were statistically significantly higher than those for group 2, while there was a small, but statistically insignificant difference between the 2 groups' written exam scores (p = .21) (Table 1).

By contrast, the mean number of videos watched was lower for group 1 compared with group 2 (3.23 ± 2.61 vs 5.70 ± 3.35, p = .011). There was no significant difference in the number of practices between the 2 groups (p = .18).

The primary aim of this study was to compare the academic performance between 2 groups of students who used different learning strategies in their PE lab classes: 1 group used a more in-person PE lab format and the other used a more online format. The PE lectures were delivered in online-only format to all students, but students had the option to choose between different lab formats: in-person lab, online lab, or mixed. We found that the majority of our students (84%) preferred in-person lab and attended more than 50% of total lab classes in person (group 1), while others used virtual learning as their primary format (less than 50% of total lab, group 2). Therefore, a further aim of this study was to explore whether the different lab-learning formats could have an effect on their learning outcomes, including both lab and lecture. Students' academic performance on OSCEs, weekly lab quizzes, and written exams were assessed. These exam results might shed light on whether students' learning outcomes could be influenced by different study formats. In the end, this study showed no significant difference in the mean scores of the lecture written exam between the 2 groups but did find significant differences in the mean scores of the lab quizzes and OSCEs between the 2 groups, suggesting the influence of different lab-learning formats on lab-related exams.

The efficacy of online learning vs traditional in-person learning has been studied extensively. The majority of studies from various levels of educational institutes concluded that there was no significant difference in student academic performance between online and traditional classroom learning with respect to format, sex, or class rank in science courses.²⁶  The authors believe that in terms of student learning, self-motivated students would be successful in either modality.²⁷  However, all these studies were about science-based lecture courses. How different modalities may affect PE lab-based courses has not been studied thoroughly. One study from a dental school that assessed the efficacy of online vs traditional face-to-face radiographic interpretation lab showed that the online lab was superior to the traditional in-person lab, which contradicts other studies.²⁸  The authors attributed such contradiction to a variety of factors, especially radiograph laboratory practical–related elements.

The current study investigated a similar topic from a different angle. We wanted to examine whether the overall attendance rate of in-person PE lab would have any impact on student performance (including weekly quizzes and written exam). Unlike the other studies, which were focused on the comparison of the exam results between online vs in-person learning, this study explored whether the correlation existed between students' lab attendance rate and their academic performance. It was our belief that more in-person lab attendance would yield better student performance on exams. However, the results were mixed. There was a positive correlation between in-person lab attendance rate and the grades of weekly quizzes, yet there was no correlation between in-person lab attendance rate and the written exam. There are 2 proposed explanations for this phenomenon. The 1st states that although both weekly quizzes and the written exam are written examinations, weekly quizzes test short-term knowledge retention, while the written exams test relatively long-term knowledge retention. More in-person lab attendance helps short-term knowledge retention. The 2nd proposed explanation is that the written exams are more focused on theoretical and conceptual understanding or, more simply, knowledge, while the weekly quizzes are more focused on hands-on practical skills. Therefore, it is not surprising to see the different outcomes. It is worth pointing out that although greater attendance of in-person labs did not help students on the written exam, it did not have a negative impact.

Another assumption of this study was that greater in-person lab attendance would result in better performance on lab OSCEs. The results of the study supported this assumption: students with above 50% in-person lab attendance did significantly better on OSCEs than those with less than 50% in-person lab attendance. This finding was interesting because most existing evidence has suggested that there is no significant difference in exam performance regardless of whether class material and exam are presented in person or online.^29,30  However, no studies have compared the difference of lab OSCE performance between in-person lab learning and online lab learning. Therefore, the finding of this study adds new information to an existing body of knowledge in relation to student academic performance in PE courses as a result of in-person or online learnings.

On the surface, this finding may be difficult to explain, that using student-centered e-learning could result in deeper learning and, therefore, yield more favorable exam performance. One study suggested that web-based video enhanced medical-student performance on PE skills.³¹  However, there are a couple of key differences between the Orientale et al³¹  study and the current study. The 1st difference is that the Orientale et al³¹  study examined the use of web-based video as supplemental material in addition to in-person lab learning, whereas the current study investigated the impact of in-person lab learning or online lab learning on student academic performance, including lab OSCE. The 2nd difference is that the former study was focused on student competence in performance PE, while the current study specifically examines student OSCE.

The difference in student OSCE performance between in-person lab learning and online lab learning could have several causes. First, online learning is not synonymous with in-person learning, which may take on a more collaborative and constructive approach,³²  especially for PE classes. Second, there are unique benefits of both formats. The benefits obtained from online lab learning might include less stress and greater flexibility for students. This may prove to be especially suitable for students with a strong self-centered learning style. Alternatively, the benefits obtained from in-person lab learning might be more engagement and deeper understanding. When students attend in-person lab classes, they obtain immediate feedback from peers and teachers. In general, this type of quick feedback helps students avoid or correct mistakes. In contrast, the feedback available from online lab class is less efficient. Students usually wait hours or longer for a peer or teacher to comment on their particular practice, a situation which does not usually arise in an in-person lab. Restricted viewing field could be another disadvantage for online labs. For example, the interaction between peers and teacher is always dynamic, while the camera is static. Sometimes such interactions might be out of the camera field, and online students might miss these interactions. Another potential explanation could be that students' preference for online format over the in-person format is not related to academic performance, but to their confidence in interaction with technology.⁷  The current study suggests that the majority of students prefer to engage in class discussions in person with teachers and peers. Making more personal contributions to in-person interaction may lead to better OSCE performance. Finally, since online labs are recorded, both groups of students can review lab recordings at their leisure. Therefore, it may have become a blended model for students who attended more in-person lab classes and thereby helped with better retention of PE class knowledge, so that they performed better on their OSCE.

The following are some limitations of the study:

1. This study is a retrospective study; a prospective cohort study will be more meaningful.

2. We were provided with very little background information for the participants other than their sex, and participants were recruited by way of a convenience sample.

3. Participants were not assigned to a group but, instead, self-selected their preferred learning group (virtual vs in-person).

4. The retrospective nature of the study leaves it particularly susceptible to an array of biases and confounders so that the conclusions should be interpreted cautiously in the absence of prospectively collected data.

In conclusion, student online lecture learning is not identical to student PE lab learning. Online lecture learning can be as efficient as in-person class learning. PE lab class contains more hands-on practice, which requires more close interaction between students and instructors. This study supports the hypothesis that more attendance of in-person lab class will help the academic performance on weekly quizzes and OSCE. Further research could aim to unpack such benefits and their relationship with student learning characteristics. Since online teaching will continue to expand worldwide in the future, those who are responsible for teaching PE lab should consider more carefully the nature and type of activities they allocate to different teaching formats.