Requiring students to justify answer changes during collaborative testing may be necessary for improved academic performance^*

Collaborative learning can refer to any instructional method in which students work together in small cohorts toward a common goal.¹  Learning strategies in collaborative learning are active and student-centered, with a variety of collaborative learning strategies being shown to promote knowledge and skill acquisition at all grade levels and in multiple professions.^2–4 

Quizzes and examinations are the most common methods of assessing academic achievement and assigning course grades. They facilitate learning assessment in a large number of students over a short period of time. The instructor is provided feedback regarding what students have learned, and students discover the scope and depth of their knowledge. Collaborative testing, in which students work together to discuss examination questions and arrive at consensus to test answers, is an extension of collaborative learning that has been reported to decrease test anxiety, and increase critical thinking, communication, and team building skills.^5,6  It also has been studied in a wide range of disciplines, including mathematics,⁷  nursing,⁸  and language training.⁹  While examining the effects of collaborative testing on academic performance, 2 chiropractic education studies reported that students involved in collaborative testing achieved higher topical exam scores and course grades compared to noncollaborative testing. However, noncollaborative final exam scores at the end of term were not significantly improved.^10,11  Woody et al.¹²  reported that, although students reported satisfaction with the collaborative model and course performance was improved as a result of participating in cohort testing, material retention was not significantly improved. In addition, students have expressed concerns that unprepared peers may obtain higher scores with collaborative exams by simply copying answers.¹³ 

In consideration of these studies, we were interested in evaluating the summative and formative use of collaborative exams in a chiropractic education program. We hypothesized that: (1) Collaborative topic-specific tests during the course will enhance student performance on a noncollaborative, cumulative final exam administered at the end-of-term, compared to a control cohort of students given traditional (noncollaborative) topic-specific exams during the course. (2) Requiring students to provide reasons for changing answers during collaborative topical exams will further enhance performance on the cumulative final exam administered at the end-of-term. (3) There will be a differential final exam question-type effect across the collaboration study cohorts (Collaborative Exam Only [CEO] and Collaborative Exam with Justification [CEJ]), with substantially greater score increases, compared to the control cohort, for comprehension versus recall questions.

The Palmer College of Chiropractic institutional review board granted this educational method study an exemption from formal review, and permission was obtained from all students to use deidentified performance assessments for this study and subsequent publications. A sample size calculation was performed for an anticipated 5%-point change in cumulative final exam scores (our primary outcome). We considered this to be a meaningful change in exam performance, representing a 0.4 effect size (Cohen's d). Consequently, the required cohort sample size (n, at 2-tailed α = .05 and β = .80) was estimated to be n = 60 students/cohort.

A total of 223 3^rd-quarter students participated in the study across 4 consecutive iterations of a 3-credit class presenting immunology and endocrinology content (January 2014–December 2014; Fig. 1). Each class offered 20 lectures of 50 minutes each. Class members in the 4 course offerings were assigned randomly, en bloc, to 1 of 2 study cohorts: (1) Control – a traditional, noncollaborative, topical exam format (1 class, n = 61); (2) CEO – a collaborative topical exam format, not requiring answer change justification (1 class, n = 82); 3) CEJ – a collaborative topical exam format, but requiring justification for any answer changes (2 classes combined, n = 43 and n = 37; totaling 80 students).

This course was offered 4 times during the academic year. The instructor, lecture format, and course content were identical in each presentation of the course. Only topical exam formats varied across the 4 classes, as described for the study cohorts above. Demographic data (sex, age, academic degrees, and ethnicity) also were collected.

Two topic-specific exams and 1 cumulative final exam were administered to each of the 4 classes taking a combined Immunology and Endocrinology course. Students in the 2 collaborative exam cohorts (CEO and CEJ) completed a topic-specific exam individually and handed in their answer sheets. Immediately thereafter, the students were allowed to form into groups of 2 to 3 students and work as a team to complete the same exam. Each student completed a second individual answer form for this second administration of the exam. In addition, students in the CEJ cohort submitted an explanation sheet, selecting from a fixed-choice list of reasons for making answer changes (Table 1). This list was compiled from a survey of students in a previous collaborative learning study by Zhang and Henderson.¹⁴ 

The 2 topic-specific exams and the cumulative final examination administered to all classes were of the single-best-response, multiple-choice format containing recall and comprehension questions. Exam questions were identical for all study cohorts. The cumulative final examination was administered individually (without collaboration) in all study cohorts.

Data were initially examined graphically to reveal underlying distribution patterns and identify outliers. We summarized and analyzed our data using SPSS version 22 (IBM, Chicago, IL). Statistical test assumptions were verified, standardized effect sizes were recorded, and means with 95% confidence levels (CI) were calculated. Study hypotheses were evaluated at the .05 family-wise α level. We applied a Mixed Design ANOVA to evaluate factorial effects on cumulative final exam scores within and across the study cohorts. After ANOVA, planned contrasts identified significant cohort and interaction effects. Significant main effects for the dichotomous “question-type” factor were identified by direct comparison of means.

Collaboration net benefit also was examined. Net benefit data were evaluated for the 2 collaboration study cohorts (CEO, CEJ) during the topic-specific exams. Collaboration net benefit was defined as the total net change in exam answers for the topic-specific exams. If a student changed a question response that was incorrect to one that was correct, that constituted a single positive benefit point (+1). If the change was from correct to incorrect, that constituted a single negative benefit point (−1). Answers that were not changed were assigned 0 benefit points.

Demographic data are summarized in Table 2. Values within parentheses are percentages. In our sample, sexes were similar in distribution across study cohorts, with a slightly greater percentage of males. Academic degree, age, and ethnicity were markedly skewed within all cohorts in favor of bachelor degrees, age < 30 years, and Caucasians.

There was a statistically significant main effect for study cohorts, F_{(2, 220)} = 5.29, p = .006, η² = .046. However, contrary to expectation (Hypothesis 1), there was not a statistically significant difference between Control and CEO final exam scores (p = .566). By contrast, CEJ final exam scores were statistically greater than the Control (p = .010) and CEO (p = .011) scores (Hypothesis 2).

Question-type had a marked main effect on final exam scores (F_{[1, 220]} = 609.77, p = .000, η² = .735). On the final exam, recall question scores were substantially greater than comprehension scores within each of the 2 study cohorts. While recall question scores were essentially the same in the Control and CEO cohorts, they were higher in the CEJ cohort. Similarly, comprehension scores were essentially the same in the Control and CEO cohorts, but were higher in the CEJ cohort. The relative increase in final exam recall and comprehension scores in the CEJ cohort compared to the Control cohort were quite similar. Therefore, contrary to expectation (Hypothesis 3), question-type did not differentially influence study cohort final exam scores (F_{[1, 220]} = .562, p = .571, η² = .005).

Of students who changed their topic-specific exam answers after group collaboration, 66.6% changed incorrect answers to correct answers (positive benefit points). Only 7.4% changed correct answers to incorrect answers after peer discussion (negative benefit points).

In the topic-specific exams, we found statistically significant main effects on collaboration benefit for question-type (F_[1,160] = 915.36, p = .000, η² = .85), and study cohort (F_[1,160] = 8.12, p = .005, η² = .048). Student collaborations produced greater net benefit for comprehension questions compared to recall questions and students in the CEJ cohort benefited more than the CEO cohort. In the CEJ cohort, the top 3 fixed-choice reasons (Table 1) for changing an answer after group collaboration were: Group members backed answers with facts (26%), Guessed first time (14%), and Misread the question the first time (14%). The remaining 12 fixed-choice reasons each constituted less than 5% of all responses, with 4 reasons each representing less than 1% of all responses: Mismatched Scantron the first time, Stress on individual test, Ran out of time, and Too many of same letter answer in a row the first time.

We expected increased final exam scores in both collaboration cohorts (CEO and CEJ) when compared to the Control cohort (Hypothesis 1). However, in our study, Control and CEO final exam scores were statistically equivalent (p = .566, Table 3, Fig. 2). This finding is consistent with that of Meseke et al.¹⁰  In a study of collaboration testing among chiropractic students, they reported that final exam scores did not differ significantly between collaboration and control cohorts. Other researchers also have reported no improvement in content retention, as measured by subsequent noncollaborative exam scores. In a college large-enrollment introductory biology class, Leight et al.¹⁵  reported higher scores for collaborative testing exams, but no improvement in content retention as measured by cumulative exam questions in a subsequent noncollaborative exam. Similarly, Enz and Frosch¹⁶  reported that collaborative quizzes were favorably perceived by students in a pharmaceutical calculations course and improved course satisfaction, but made no improvement in a noncollaborative mid-term and final exam.

In contrast to our finding of equivalence between the CEO and Control cohorts and the studies cited above, some investigators have reported that collaboration testing significantly improved subsequent exam performance. Rao et al.¹⁷  reported that collaborative testing in a college physiology class enhanced student understanding of the course material. However, the validity of this conclusion may be challenged because their learning assessment exams all included a 20% inclusion of the collaborative testing scores. In a randomized crossover study incorporated into an undergraduate exercise physiology course, Cortright et al.¹⁸  evaluated the academic impact of collaborative testing using 1-question multiple-choice quizzes administered at 2 to 3 lecture pauses during each 50-minute lecture. They reported that collaborative testing produced increased mastery of original material and improved ability to solve novel problems with the information learned. However, review of their study methods revealed that this improved learning was actually similar to our “collaboration net benefit,” comparing student answers on the same question before and after collaborative testing. There was no reference to noncollaborative examinations evaluating a residual improved learning effect.

It is noteworthy that, in our study, the CEJ study cohort final exam scores were greater than the Control (p = .010) and CEO (p = .011) cohorts (Table 3, Fig. 2). Our Hypothesis 2 was that justifying answer changes (CEJ) would enhance the improved final exam performance obtained by collaboration alone (CEO). This was not supported by our findings. Rather, the differing final exam outcomes for CEO and CEJ versus Control suggest that requiring justification of answer changes during topical test collaboration was necessary for subsequent improved final exam performance, not simply an enhancement of a CEO cohort effect.

Question-type demonstrated a statistically significant main effect (p = .000) with substantially greater scores for recall compared to comprehension questions in the final exam (Table 3, Fig. 2). This is not surprising; final exam scores would be expected to be highest for the least challenging test questions (recall questions). Also, as would be expected, beneficial answer changes during topical exam collaboration was greatest when answering comprehension questions, the more difficult exam question-type (p = .000, η² = .85). Students in the 2 collaboration study cohorts obtained a positive topical exam benefit 9 times more frequently than a negative benefit; 66.6% changed answers from incorrect to correct while 7.4% changed answers from correct to incorrect. However, the differential net collaboration benefit during topical exams did not produce a similar differential effect between question-type and study cohort effects in the cumulative final exams (p = .566). Recall and comprehension final exam score trends across the study cohorts were statistically equivalent.

Several mechanisms may mediate enhanced performance during collaborative testing. First, collaborative testing may reduce test anxiety by reducing the sense of competition and being part of a group may give a sense of security. Anxiety has been shown to interfere with learning and effective test taking.^19,20  Second, it has been reported that a collaborative environment encourages students to become active learners, improves learning attitudes, and enhances critical thinking skills and depth of understanding.^21,22  Lastly, collaborative testing provides an opportunity to discuss reasons for a particular answer, requiring deeper consideration of the course material. In addition to evaluating their own reasoning with regard to test questions, students may identify and fill in knowledge gaps. Chi et al.²³  note that when students explain their reasoning to classmates, “the learner becomes a teacher.” Michael and Modell²⁴  state that “teaching requires generation of explanations, both for oneself and for the learner.” In our study, we examined whether the putative beneficial effects of 2 collaborative topical exams administered during the academic term would have long-term benefits on academic performance assessed by a noncollaborative, cumulative final exam administered at the end of the term. In addition, we examined whether any observed long-term benefit might be different for students in a CEO cohort, having a collaborative topical exam format, but not requiring answer change justification, versus students in a CEJ cohort, also having a collaborative topical exam format, but requiring justification for any answer changes. Students in the CEJ cohort were required to select a fixed-choice reason when changing an answer (Table 1). The most frequent answers provided by CEJ students suggested that this additional task encouraged additional consideration of the exam questions, rather than simply copying answers from cohort classmates.

We concluded that test collaboration with the requirement that students explain their reasons for making answer changes is a more effective learning tool than simple collaboration that does not require answer change justification. Moreover, this effect during collaboration tests is greater for the more challenging comprehension-type questions than simple recall questions. Finally, we found a long-term influence on learning, as reflected by higher noncollaborative, cumulative final exam scores. However, this benefit was observed in the CEJ cohort, not the CEO cohort, suggesting that requiring students during collaborative exams to explain their reasons for making answer changes may be necessary for a long-term academic benefit.

This work was funded internally. The authors have no conflicts of interest to declare relevant to this work.