Effect of unproctored versus proctored examinations on student performance and long-term retention of knowledge

Tests are a typical practice used to gauge student progress. The traditional testing format in higher education is a proctored, closed-book, pencil-and-paper exam.¹  The key argument in favor of this kind of test format appears to be that it reduces the likelihood that it will be tainted by unethical student activity.^1,2  With the emergence of newer technologies, online exams are being used more frequently by instructors in on-campus and online courses.^3,4  The advantages of online testing are increased grading accuracy, shorter grading times, and the ability to provide students rapid feedback.⁵ 

Tao and colleagues⁶  point out that instructors have 2 options for online testing: (a) students can take an online test in a proctored setting, or (b) students can take an unproctored online take-home test setting at a computer of their choice. They indicated that the biggest advantage of computerized take-home tests is the convenience to instructors and students. However, aberrant student behaviors such as cheating still present a substantial challenge for instructors who implement take-home testing. Reports on the importance of proctoring in the administration of online exams among different majors have been mixed. In a course, Introduction to the Computer in Business, Hollister and Berenson⁷  found no significant difference in students’ academic performance results between proctored and unproctored online exams. Also, they did not find evidence of cheating behavior in the unproctored section. Frein⁸  compared test scores between proctored paper-and-pencil in-class tests, proctored online tests, and unproctored online tests in an introductory psychology course and found no significant differences. However, differences between proctored and unproctored tests have been reported by other researchers. Schultz and colleagues⁹  compared student performance in proctored paper-and-pencil tests with that of unproctored online exams in marketing, management, and accounting subjects and observed that the unproctored exams resulted in noticeably better test scores. A considerable performance gap between proctored and unproctored tests in a psychology course was also noted by Carstairs and Myors.¹⁰  On a proctored paper and pencil exam, students’ average scores were 5 points lower than on an unproctored exam. Rakes et al¹¹  tracked the performance of 114 master’s and doctoral level students who completed an online unproctored exam or a conventional proctored exam. In contradiction to the findings from Schultz’s study and Carstairs’ study, they found that participants in the traditional proctored test performed significantly better than those who participated in the online unproctored test.

Unproctored online assessments raise concerns about test scores being exaggerated if students are permitted to consult course materials.¹²  Two studies reported that unproctored online tests had more instances of cheating than proctored tests.^13,14  Although multiple studies reported that cheating was common in online courses, few studies really assessed cheating behavior. According to Corrigan-Gibbs et al,¹⁵  several researchers found evidence of considerable cheating in online tests, while others did not.^16,17 

Due to the inconsistent outcomes from earlier studies, further evaluation of proctored and unproctored online test performance is warranted. We hypothesized that: (1) Students who took unproctored exams would have better test grades than those who took proctored exams; (2) Students who took unproctored exams would spend more time on exams than those who took proctored exams; and (3) The unproctored and proctored test formats would affect long-term retention over a period of 10 months after initial learning. To determine whether the proctoring format had an impact on students’ academic behaviors, this study contrasted students’ performance on tests with and without proctoring, including test-taking time, and long retention of presented class information.

Prior to implementation, the study protocol was reviewed and approved by the Palmer College of Chiropractic institutional review board (Assurance #: X2021-001).

The academic year on our campus is organized into 4 terms: fall, winter, spring, and summer, according to a quarter system. Four sessions of the endocrinology course were provided in a single academic year (October 2020 to September 2021) with the same instructor who provided the same lecture structure, and course materials.

This was a retrospective study. Preexisting assessment data were obtained for 234 third-quarter students enrolled across the 4 consecutive presentations of the same endocrinology course. These 4 classes were divided into 3 testing groups, including unproctored, proctored (2 classes), and mixed based on the format of the midterm and final exams (Table 1). Students taking unproctored exams were informed that tests were to be taken without written or verbal references during the test; however, there was no formal way for instructors to verify this. For data analysis, unproctored midterm exam score data were combined across 2 classes (n = 86) for comparison with proctored midterm exam data combined across 2 classes (n = 148). Unproctored final exam score data were obtained from a single class (n = 26) while proctored final exam score data were combined across 3 classes (n = 208).

The online testing platform, Brightspace (D2L Corporation), was used to administer each exam. The midterm exam and comprehensive final exam were administered in weeks 5 and 11 of each academic term, respectively. Each exam consists of 50 multiple-choice, single-best-answer questions. Students were allowed 50 minutes to complete each exam. The proctored or unproctored exams were chosen based on the condition of whether the exams were taken on campus or remote due to the COVID-19 pandemic. There was only 1 class, termed a mixed class, whose midterm was unproctored and final exam was proctored in the same academic term.

In the sixth quarter, about 7 months after the final exam, all students were requested to take a voluntary exam called the “recall exam.” In all classes, the recall exams were the same and unproctored. Without supervision, these students completed their recall exam. The mixed class’ students skipped the recall exam. The students’ grades in the “recall exam” were not part of their final grades of that academic year.

Results were analyzed using SPSS version 22 (IBM Corporation). The independent t test was used to compare the impact of proctored/unproctored exams on grades for midterm exams and for final exams, as well as for time taken during these exams. In a secondary analysis, the difference between each testing group’s midterm and final exam scores and test duration was analyzed by the paired t test (for each test format: unproctored, proctored, and mixed). A percentage scale with a minimum score of 0% and a maximum score of 100% was used to standardize all exam results. As a predictor for the recall exam scores, we conducted a multiple linear regression analysis on the final exam score. The examination format as an independent variable was entered into the regression. Multiple linear regression requires at least two independent variables, which can be nominal, ordinal, or interval/ratio level variables. For all quantitative measures, effect sizes were estimated, and 95% CIs were reported. Statistical significance was evaluated at p = .05.

The total number of students included in this study was 234 (97 women, 137 men). The majority of students in all 4 classes were Caucasians (68%), less than 30 years old (91%), with undergraduate degrees (96%). Very few have graduate degrees.

Independent t tests showed the midterm exam mean scores were significantly impacted by the exam format, t (₂₃₂) = 5.25, being higher in the unproctored group compared to the proctored group (p < .01). Similar results were also observed in the final exams in that unproctored students received higher grades compared to those in the proctored group, according to an independent t test; t (₂₃₂) = 2.13, p = .01 (Table 2).

Additionally, a paired t test revealed that the mean scores of the final exam in proctored and mixed classrooms were significantly lower than the corresponding midterm examinations (p < .01) in both cases. When compared to the midterm exam, the mixed class’s final exam mean score fell by 16% (Fig. 1).

When we compared the time it took students to complete their midterm exams in the unproctored group (n = 86) and the proctored group (n = 148), we observed that the unproctored group’s mean time (M = 40.7, SD = 10.2) was 24 minutes (144%) longer than the proctored group’s (M = 16.7, SD = 7.0); t (₁₃₂) = 19.4, p < .01. Similar results were obtained when we compared the amount of time students spent on their final exams. Students in the unproctored group (M = 47.0, SD = 8.8) spent considerably more time on final exams than those in the proctored group (M = 21.5, SD = 9.0); t (₃₇) = 17.5, p < .01.

A paired-samples t test was performed to compare the length of time spent in midterm exam and in final exam in each group (mixed, unproctored, and proctored). In the proctored group, the results indicated that the final exam time (M = 20.6, SD = 8.4) was significantly longer than the midterm exam time (M = 16.7, SD = 7.0); t (₁₄₇) = 8.8, p < .01. By contrast, in the mixed group, the final exam time (M = 23.7, SD = 10.0) was significantly shorter than the midterm exam time (M = 38.8, SD = 10.4); t (₅₉) = −9.7, p < .01 (Fig. 2).

Twenty (77%) students from the unproctored class and 76 (49%) students from the proctored classes completed the recall exam.

Independent t tests showed a significant difference between the unproctored (M = 60.2, SD = 14.7) and proctored M = 88.1, SD = 8.2) groups recall test scores (t (₉₄) = −4.1, p < .01). The final exam was shown to be a statistically significant predictor of the recall exam, accounting for 28% of total recall exam scores (p < .01, R² = 28.3%, ß = .53, step 1 of the regression model). The addition of the examination format, into the regression model significantly increased predictive power (p < .01, R² = 39.8%, ß = .68, step 2 of the regression model).

It has been debated in the past whether proctored or unproctored tests could affect students’ academic performance. According to the results of our study, students who took tests without a proctor had higher test scores, longer exam times, and lower repeated test scores 7 months after the final exam. Comparing the 3 categories to the students who used proctored tests, all 3 showed statistical significance.

The conclusion that students in the unproctored group scored much better than students in the proctored group in terms of grades agrees with other earlier research.^7,18,19  The score difference between unproctored versus proctored exams differed by 5.46% on the midterm and 4.84% on the final exam. These differences were also found within each class. In other words, the average score decreased by 16 points when the same students switched from an unproctored midterm to a proctored final. Furthermore, students spent significantly more time on unproctored exams compared to proctored ones on average. This supports our hypothesis.

In the current study, the students with unproctored exams were informed that tests were to be taken by themselves without using any resources or discussion with peers. But there were not any monitoring methods, such as proctoring software, used during the exams. On average, the time spent on taking the midterm exam and the final exam was different. In all unproctored exams the average time spent on exams is significantly longer than on proctored exams. It is likely that the better grades and longer exam-taking time are interrelated. Unproctored exams created an opportunity for some students to seek answers from other sources during exams. In other words, they took extra time searching for the answers. Due to such a benefit, the extra time they invested might have improved their grades.

There are several other factors, such as question type, which may explain why students performed better on unproctored exams than on proctored exams. In this study, the question structure was the same for both proctored and unproctored tests. Varble’s²⁰  research indicates that topics on tests fall into 1 of 4 categories: (a) recall, (b) understand, (c) analyze, and (d) apply. Unproctored students may prepare less for recall-type questions because they can check their responses by consulting reference material which also increases the time spent answering questions. According to a prior study, it is easier for students to cheat on these questions than it is for them to understand, analyze, and apply them. All exam questions in the current study were multiple choice, and many of them were recall-type inquiries. The student might benefit much from attempting to look up these questions while taking the test. According to some studies, questions that call for certain abilities to be used, such as math calculations, would not show a significant grade difference between proctored and unproctored tests.^7,21 

According to Hilton et al, students who believed they were being monitored by online proctors felt they could not behave inappropriately throughout the exam, but those who believed they were not being watched collaborated with their peers for some questions.²²  Numerous studies have found that there was a probability for academic dishonesty and a perception that cheating occurred more frequently in online classes.^23–26  Researchers discovered evidence of massive test-taking fraud online.¹⁵  Although this study did not evaluate cheating behavior, given the statistically significant disparities in exam length and scores between the 2 proctoring conditions, we could not completely rule out the potential that some students may have

Of course, the better grades for unproctored students are not always attributed to extra help and longer time, it could also be due to the testing environment. Some students prefer an isolated, quiet, and less distracting environment when taking exams. The distance unproctored exams just provide such an environment that may make students feel less anxious and in turn boost their performance.²⁷  In a prior study, we discovered that 166 third-quarter chiropractic students (85%) reported having moderate to high test anxiety.²⁸  Additionally, we noticed a statistically significant inverse relationship between test anxiety and performance on written exams. It is possible that unproctored exams create a less stressful atmosphere and lower the students’ degree of test anxiety and, as a result, increase their performance due to a more comfortable exam environment.

The study’s final finding was that on the recall exams given 7 months after the final exams, proctored students outscored unproctored students by 20 points, who had performed better on the middle and final exams. This begs the question of whether various students’ preparation strategies might affect their ability to remember knowledge over the long run. According to several studies, students who were aware that they would be evaluated via a take-home, unproctored exam tended to study less diligently than they would have if they were evaluated via an in-class, proctored exam.^29–32  It is highly likely that because they were proctored, they had to work harder to prepare, proctored students in our study may have greater long-term information retention for that reason.

Finally, the probability of academic fraud due to an unproctored circumstance should not be discounted. This conclusion is crucial because it might help to resolve the question of whether proctoring all exams is necessary to ensure academic integrity. The options to preserve academic integrity might be to reduce test duration, use varied question types, use a lockdown browser, use questions on tests that are chosen at random, and prohibit going back.

There are limitations to this study. First, because it was a retrospective study, there was a disparity in sample sizes between proctored and unproctored groups. Compared to proctored group and mixed group, the sample size for the unproctored group was 82% and 56% smaller, respectively. A sample size that is too small could have an impact on statistics. Second, this was 1 course (endocrinology). This course requires technical terms to be memorized and accurately applied, and where assessment only included multiple choice questions. It is unclear if the effect would be as significant in courses without timed, closed-ended assessments. Finally, the recall exam was not a formal exam. The students might not take it as seriously as formal exams. In addition, the results of the recall exam may not be accurate because it was unproctored. Therefore, the grades could be only used as reference.

This study sheds light on the impact of proctoring on students’ testing outcome. The findings corroborate our hypothesis that students spend more time and receive higher scores on unproctored exams, which is consistent with prior research. Additionally, it revealed that the unproctored cohort experienced poorer long-term recall. These outcomes were likely explained by differences in exam preparations.

The authors have no conflicts of interest to declare relevant to this work.