Comparison of student satisfaction, perceived learning and outcome performance:Blended instruction versus classroom instruction

Online and blended instruction is gaining popularity within many facets of the US educational system.^1–3  In chiropractic education at the University of Western States, the traditional lecture/face-to-face classroom model remains the predominate mode of instruction. Chiropractic students at the University of Western States average 33 hours per week of instruction during their first 2 years, almost all of which is face-to-face lecture/lab format. The goal of this study was to investigate whether the traditional classroom lecture could be replaced with blended instruction to achieve equal or improved learner satisfaction and outcome performance.

Several studies have suggested that online courses and blended instruction can be as effective as the traditional classroom mode of instruction.^2–8  A meta-analysis conducted by the US Department of Education identified more than 1000 empirical studies of blended and online learning from 1996–2008.⁶  This meta-analysis concluded that, on average, those receiving online instruction performed modestly better than those receiving face-to-face instruction.⁶  Other studies have shown that students taking a blended course of study can perform as well or better than their traditional classroom cohorts.^2,4  Studies have also focused on learner satisfaction with online learning. These studies have also shown that students who have taken online courses versus classroom instruction report better course satisfaction.^1,4,7,9 

Creating blended access to course material enables flexibility in student schedules with a positive impact upon student satisfaction and perceived learning.^6,10,11  A basic course entitled Radiographic Technique I was chosen to be the subject of this investigation. This course was chosen because it combines an equitable mix of didactic knowledge and practical skills demonstration. Specifically, the course focuses on content in the following areas: radiologic physics, radiation health and safety, and clinical radiography. This course is 10 weeks in length with 4 hours of lecture per week and a 1-hour hands-on practical laboratory.

One of the primary goals in the design of this project was to determine whether the lecture component of the course could be replaced with a blended format without compromising satisfaction or performance on objective measures of competency. The hands-on laboratory section of this applied science course allowed students to apply what they have learned in the lecture portion of the course. Students attended a 1-hour practical laboratory session per week to apply and master the concepts provided in the blended course material through laboratory exercises. This hands-on lab experience allowed students to interact with each other and with the course instructor.

It was hypothesized that learner satisfaction with blended instruction would be of equal or greater levels compared with traditional face-to-face methods. It was also hypothesized that objective measures of competency would be equal or greater with blended instruction as compared with traditional classroom instruction. We desired to see if students taking blended instruction would report greater flexibility in their schedules than traditional face-to-face learning students and if all students would find that a weekly hands-on practical laboratory session with the course instructor and other students was helpful to their learning.

All participants in this study were 2nd year chiropractic students enrolled at the University of Western States located in Portland, Oregon. The institutional review board at University of Western States reviewed and approved this project prior to initiation of the project. A total of 122 participants signed an informed consent form prior to being randomly assigned into 1 of 2 cohorts, 1 cohort taking the blended lecture component of the course and the other cohort taking the traditional face-to-face classroom course. Both cohorts were assigned a weekly hands-on practical laboratory session led by the course instructor. All laboratory sessions contained a mixture of students from both cohorts.

The traditional classroom cohort's learning experience was similar to how the course has been historically taught at the university. All course materials assigned to this cohort, including the course syllabus and note packets, were provided to students on the first day of class. No significant course modifications had been recently made. The classroom cohort was not given access to the blended course.

It is important in the development of a blended course to have clear learning goals and objectives, keep the students engaged during learning sessions, and make the course enjoyable.¹²  The design and implementation of this online course was based upon educational strategies used to promote the online learners' experience from the Quality Matters Rubric Standards of 2008–2010.¹³  All lecture material for the blended cohort was provided in the online course. The course syllabus and course notes for the blended group were also provided in an electronic format within the course. Technical requirements required text information, and a course map/tutorial were also made available to students in the blended course. Course instructor information and the opportunity to contact the instructor for questions and feedback were also provided within the course.

In blended instruction, students would benefit by having familiarity with the learning platform.¹⁴  Moodle 1.9 (Moodle Pty Ltd, West Perth, WA) was used as the educational platform for the blended course. Students participating in this project had prior experience using the Moodle learning platform in other courses at the university prior to enrollment for the course. Tegrity lecture capture technology (McGraw-Hill, NY, NY) was used to record the course lectures, which were then integrated into the Moodle course. The blended cohort was asked not to attend the classroom lectures.

The blended course was divided into 10 weekly modules based on the term schedule. Each weekly module contained lecture material for the course and was divided into 2 categories: radiographic physics/health and safety and clinical radiology. Information regarding the laboratory session for that week was also available online, although all students were required to attend the laboratory section of the course. Course topics within each module were distributed into smaller segments, which were referred to as “lessons.” Lecture material associated with the course topic was then recorded by the course instructor using Tegrity recording software. An effort was made to keep the video lecture for each lesson to 10 minutes or less to keep the student engaged, although this was not possible for all topics. A video presentation covering lecture material, coinciding course notes, and a short quiz to reinforce the material was provided for each course lesson through Moodle. The quizzes were not calculated in the overall grade; these were strictly formative activities to reinforce the blended course content.

Students assigned to the blended cohort were given access to the online material while the traditional classroom cohorts were denied access. Students, regardless of the mode of instruction, were expected to pass the course with a grade of 70% or higher. Both cohorts were tested together in a classroom setting at midterm and at the end of the course. Another comparison of the study was to measure final grade outcome using the midterm and final exams. No further opportunities were provided for grade enhancement. Measurement of grade averages of entire cohorts and not individual test performances were done to protect students' identities. As part of the study, each participant was required to complete an anonymous survey before taking their midterm and final exam (Fig. 1). This survey was administered to both cohorts prior to the midterm exam and, again, prior to the administration of the final exam. Student anonymity was ensured. Anonymity of individual performance, within the 2 cohorts, was ensured at the conclusion of the course, and prior to analysis, through a de-identification process that completely separated each student from any identifying information including name, registration information, or self-chosen identification.

The survey instructed participants to provide honest feedback about their learning experiences. The survey consisted of a series of questions that included student demographics (Table 1) and 9 closed-ended questions that included learner characteristics, perception of learning, overall satisfaction with the course, and if their experience was an effective way to learn (Table 2). Two survey questions were open-ended that elicited respondents' opinions about strengths and problems and limitations of their learning experiences. The survey data from this study are based upon comparing students' experiences taking an asynchronous online blended course versus taking a traditional classroom course.

The students were asked to respond to questions or statements based on which cohort they were assigned using the following options: (1) strongly dissatisfied/disagree, (2) moderately dissatisfied/disagree, (3) slightly dissatisfied/disagree, (4) slightly satisfied/agree, (5) moderately satisfied/agree, (6) strongly satisfied/agree, (0) no opinion. These values were used to generate the data provided for survey questions responses in Table 2 and adjusted means of students' experiences in Table 3. All survey responses with a numerical value of 4, 5, or 6 were considered as satisfied/agree.

The statistical software used for this project was Stata 11 (StataCorp, College Station, TX). Linear regression was conducted to establish mean differences, confidence intervals, and determine p values. Survey questions were adopted from existing, validated, and peer-reviewed surveys. In addition, the survey used in this study went through a thorough peer-review process as well as evaluated for its face validity. Anonymity of individual test results was ensured at the conclusion of the course, and prior to analysis, through a de-identification process. Test results were organized by assigned cohort and names were removed prior to analysis making it impossible to link the student to their individual test scores. The list that links the student to the student identification was retained by the course instructor. Access to the list was restricted and was only available if directed by the institutional review board or by Oregon or Federal Statute.

The open-ended survey questions asked the respondents if they had any further comments regarding strengths and weaknesses of the course based on their cohort. Content analysis was conducted by reviewing each response and establishing a code based on key words or phrases. Codes were grouped into categories and lastly categories were grouped into overarching themes.¹⁵ 

One hundred twenty-two (122) students participated in this study. A total of 62 students made up the blended cohort of the study, 45 from winter term and 17 from spring term. A total of 60 students participated in the classroom cohort, 45 students from winter term and 15 from spring term. The lower sample size in the spring term was due to an overall smaller class size. The number of students enrolled in the class who chose not to participate in the study was comparable from winter to spring based on class size.

Table 1 shows the distribution sample for winter and spring terms as well as the combined total of participants assigned to each cohort. Table 1 also summarizes participant characteristics and students' attitude toward blended courses prior to enrolling in this project. The demographic information (age, gender, prior blended experience, incoming computer skills, and prior attitude regarding blended courses) was taken from the midterm survey.

Both blended and classroom cohorts were examined together in a classroom format for both the midterm and final. The course average for the blended group was 86.3% compared with the classroom group's average of 86.7%. This demonstrated that students taking the blended format could perform comparably with traditional classroom instruction.

According to the survey, approximately 78% of participants were between the ages of 20 and 29, whereas 22% were over the age of 30. Seventy-four percent of participants reported that they had taken 1 or more blended courses, whereas 26% had never taken a blended course. Over 95% of students rated their computer skills as medium to high (Table 1). The participants' attitude toward online courses at midterm and final showed no preference for either blended or classroom instruction for either group (Table 1).

Students in both the blended and classroom cohorts agreed that they were satisfied with the delivery method of the course and felt that the delivery method was effective, although students in the blended group reported an overall greater satisfaction over the traditional classroom cohort in many aspects.

There was no significant difference between cohort groups regarding satisfaction of course delivery method and its effectiveness. Both blended and classroom cohorts were overall satisfied with the course delivery method that they were assigned and agreed that the delivery method was an effective way for them to learn. Students in the blended cohort reported an overall greater satisfaction (90%) and effectiveness (87%) over the traditional classroom cohort satisfaction (78%) and effectiveness (75%) at the final. When questioned about whether the course delivery method was an enjoyable way for them to learn, more students in the blended group (82%) agreed that the course was an enjoyable way for them to learn as opposed to the classroom cohort (70%) at final. When asked if similar instruction methods should be used for the course in the future, 88% of the students in the blended course agreed that similar instruction should be used for the course in the future compared with 62% of the classroom participants. When questioned whether the course took more time and effort than it was worth, 68% of participants in the classroom cohort agreed that it took more time and effort than it was worth as opposed to 42% in the blended group. There was also no significant difference (p = .67) in how well students felt like they were prepared going into their final exam (Table 3). An error in the delivery of the winter final survey inadvertently omitted questions regarding laboratory activities, although all other surveys given to participants contained these questions. Despite this error, over 92% of students surveyed in both groups agreed that the laboratory activities were an enjoyable way to learn and found that the laboratory activities were helpful in their learning process. Our original hypothesis stated that “all” students would find that this activity would prove beneficial to their learning, although 8% of those in the study answered negatively to the survey question or had “no opinion.” When participants of both groups were asked whether they preferred blended learning with laboratory activities over the traditional classroom learning with laboratory activities, the blended group was more in agreement (87%) than the classroom cohort (75%) that they would prefer blended instruction with laboratory activities. There was more of a mixed response to this question from the classroom group (Table 2). The classroom cohort reported that they spent on average 2 more hours per week on the course than the blended group at the midterm survey and 1 more hour per week on average than the blended group on the final survey (Table 2).

Content analysis indicated 2 themes and categories within themes. The number of responses based on codes of each category was reported.

Students assigned to the blended cohort reported that strengths of the course gave them more flexibility in their schedule and allowed them the ability to learn at their own pace. They also found that online quizzes were helpful. These students also reported the weaknesses of the course. They reported that they tended to procrastinate and felt like they were missing out on important points made in the classroom lecture and could not ask the instructor questions on the spot (Table 4).

Students assigned to the classroom cohort also reported strengths and weaknesses of the course. The students assigned to the classroom cohort reported that they liked the face-to-face interaction with the course instructor and other students. The students also liked that they could ask questions and have them answered immediately by the instructor. Students also reported that they liked that the course was already part of their weekly schedule, whereas others complained that there was no flexibility in their schedule. The classroom students reported that they felt at a disadvantage because they did not have access to the online quizzes, although similar quiz questions were available at the end of each chapter in their required text. Students had adequate access to the course material through the library. Students were also dissatisfied that they were required to learn at the pace of the instructor and the lecture schedule (Table 5).

This study demonstrates that students taking a course in a blended format can have a greater sense of satisfaction and perform just as well as students taking the same course in a traditional classroom environment. Although there was no difference in the students' satisfaction in the delivery method of instruction, the blended students reported that they had a better overall course experience. The blended format of learning offers students more flexibility in their schedules and allows them to learn at their own pace. Students spent “2 hours less per week” than the traditional lecture style cohort. The outcome performance between groups showed no significant difference in their final course grade. Therefore, blended instruction can be implemented into traditional educational programs that can help students balance their schedules, have a positive impact on their perceived learning, and allow them to achieve the same outcome success as the traditional classroom format of instruction.

There was a lack of enforcement to ensure that the blended students were accessing the lecture material on a weekly basis that kept pace with the classroom cohort. There were no protocols in place to control whether blended students attended the classroom lectures. Although classroom students were blocked from accessing online material, there were no control measures taken that would restrict the classroom participants from accessing the online material through their classmates who had online access. Because students were already enrolled in the program and placed into specific smaller cohort groups, we were unable to separate blended vs classroom students in the lab sections. This allowed students from both groups the potential to share information about the lecture portion of the course. This may have led to possible peer-to-peer teaching among the students, which could have had a potential effect on the objective data of the study. However, the students seemed to favor the blended learning experience based on the subjective data.

Due to the number of students in the blended cohort who reported procrastinating, accountability or benchmarks implemented into a blended course would be helpful. This should be done with little impact on students' flexibility as possible while ensuring that the students are keeping pace with the course schedule. For future studies, implementation of tighter controls should be put into place to ensure that there is no crossover of student's access to each other's learning environment. It would also be beneficial to offer formative assessment quizzes to the classroom cohort as were made available to the blended cohort. Future studies should look at ways to keep the lab groups separate during these activities to avoid crossover of information between students from different groups. Access to interactive student to student and student to instructor communication should be implemented to facilitate immediate questions and concerns of students taking a course in a blended format.

Based on this study, further research needs to be done to investigate how blended instruction can be implemented into traditional educational programs that can help students balance their schedules, have a positive impact on their perceived learning, takes less time than classroom instruction, and allow them to achieve the same outcome success as the traditional classroom format of instruction. The authors feel like this study, considering the limitations, can be a useful tool to base further studies on blended learning.