Acceptability and Perceived Effectiveness of Web-Based Self-Instruction in Clinical Orthodontics

The dental profession in general, and orthodontics in particular, is facing a serious educational problem. Schools are having great difficulty in retaining full-time faculty and finding replacements for those who leave or retire.1,2 Although all parties agree that something has to be done, precisely what to do remains under study by a variety of appointed task forces.

Two complementary possibilities that can be considered are increased use of self-instruction to replace lectures and greater use of distant faculty through the Internet, both of which can enhance faculty resources. In dental education, self-instruction with appropriate teaching material has been shown to have a positive effect on student achievement,3 and it has been documented that computer-assisted learning (CAL) is as effective as other methods of teaching and can be used as an adjunct to traditional education.4 For orthodontics, CAL is not new, and data exist to show that it can be well received and effective at both the predoctoral and graduate level.5–7 Komolpis and Johnson8 recently concluded that teaching orthodontic diagnosis using digital records on a Web site was as effective as using conventional records. Self-instructional programs can be used in preparation for discussions in small-group seminars, which are reported to be a particularly effective way to teach concepts of clinical practice.9 Botelho10 has reported that the learning experience of CAL in restorative dentistry is enhanced when small-group discussion is used in combination with a CAL program, and it seems likely that this would also be true for orthodontics.

This article describes the development, implementation, and evaluation of a self-instructional Web-based course in clinical orthodontics for dental students in their final years, using structured small-group seminars for discussion and review. Both the teaching material and the instructional method also would be applicable to advanced general dentistry and orthodontic specialty training.

In a new course for the fall of 2003, 18 lecture hours of instruction in advanced clinical orthodontics, the final unit in the predoctoral orthodontic sequence at the University of North Carolina (UNC), was replaced with Web-based self-instruction and small-group seminar discussions. The broad-course objectives remained as they had been previously: to provide students with the ability to determine orthodontic treatment needs in children and adults, to recognize treatment alternatives for orthodontic problems, and to provide appropriate counseling to the patients regarding their orthodontic needs. The course was modified, however, to place an emphasis on interaction with orthodontists in consultations and referrals of patients with the more severe problems. It included an in-depth review of indications and contraindications for extractions in the treatment of both Class I crowding/protrusion and Class II/III camouflage, indications for early vs late treatment, interactions with orthodontists in complex adjunctive treatment, and appropriate communication between the family dentist and the orthodontist in both directions.

The revised course consisted of one introductory lecture, 10 self-instructional modules, reading assignments on the subject of each module, and four small-group discussions. The computer modules incorporated images, text, video clips, and self-tests with detailed answers to the questions. They were produced in Macromedia Director. The modules were placed on the UNC dental school Web site (Figure 1), where students could view them on their own computers after downloading the (free) Macromedia Director Shockwave program. This permitted student access from any location that allowed them a high-speed connection to the site. High-speed access is available everywhere on the UNC campus, as it is now at most universities, and many students already have it at home. Dial-up connection was possible, but students were told that the graphics would be very slow with it.

The course was scheduled for a nine-week period. After the introductory lecture, three of the instructional modules were assigned to be studied during the next two weeks before a seminar to discuss the material. For the seminars, the students were divided into six groups, and a third-year orthodontic resident served as the seminar leader. A seminar outline in PowerPoint that included clinical examples was prepared by the course director and reviewed with the seminar leaders, so that the same material was emphasized and discussed in each seminar. This was repeated for the remaining modules. Four seminar sessions covered the material in all 10 modules.

After each seminar, each student was asked to evaluate the teaching modules in that unit and the seminar using a 10-point Likert scale (Figure 2) to rate its acceptability—how well did you like it?—and perceived effectiveness—how well did it help you learn? At the final examination, they also were asked to rate the entire course. The average response rate for the modules and seminars was 73%, with a range of 60% to 79%. For the entire course, 95% of the students returned an evaluation. The effectiveness of this instructional approach also would be evaluated by performance on the final examination and by later feedback from the clinical faculty.

Table 1 shows the median and ±25% data for the acceptability and perceived effectiveness of the 10 modules and four seminars. For the modules, the median acceptability score on the 10-point scale was nine for all but one (which was 8.5), and the perceived effectiveness median score was nine for all. All the seminars had a median of nine for both attributes. The high median scores for both attributes and the high percentage of students who ranked the modules and seminars as excellent or good (Table 2) show that the great majority of the students liked them and felt that both were educationally quite effective. On the evaluation sheets (Figure 2), favorable comments outnumbered unfavorable ones by 5:1. Summary scores for the modules, seminars, reading assignments, and the one lecture in the course are shown in Figure 3. All were rated as quite acceptable and effective, with no statistically significant differences.

Although the residents who served as seminar leaders were specifically instructed on how to conduct each seminar and worked from the same PowerPoint screen show, differences in ratings related to the instructor's style and personality were anticipated. As shown in Table 3, the differences were small and all the seminar leaders were rated quite positively. The structured seminar outline and presentation of the same clinical examples in each seminar seemed successful in controlling variations among the small-group sessions.

The percentages of the students by their own judgment of competence after the course are shown in Figure 4, comparing the 2002 course with lectures only with the 2003 course with the self-instruction/seminar method. The 2003 course content was changed to place greater emphasis on clinical evaluation of patients and assessment of what appropriate orthodontic treatment would be, and the 2003 students, who almost surely knew more about determining the orthodontic patient's needs than their predecessors, were significantly less likely to be at either extreme of confidence in doing this (P < .05). For the students' judgment of their ability to recognize treatment alternatives for the orthodontic problems and provide appropriate counseling, the differences were not significant.

Questions on the final examination for the course were different in 2002 and 2003 because of the change in emphasis within the course. The 2003 examination, like those in the past, resulted in a typical bell-shaped curve of student performance, ranging from excellent to required remediation. Will clinical faculty observe a difference in the way the students who had undergone the new course evaluate orthodontic problems? This remains to be determined.

This study supports previous findings that Web-based self-instruction and small-group discussions are well accepted by students as an alternative to lectures. They consider it an effective method to learn. How effective it really is must be determined by performance in examinations and, more importantly, in clinical management of patients, but it appears that for clinical orthodontics as for other areas of instruction,8 this approach is at least as effective as traditional lectures. Our results suggest that the dental students who had undergone the revised self-instructional course may be more realistic about their ability to evaluate orthodontic problems than their predecessors, but it has not yet been possible to see if there are differences in clinical performance.

Self-instruction using computers is still considered a novel educational method that can create a positive response in the students' perceptions just because it is new.4 In our study, familiarity with computer use was not a factor because these students had previous experience with CAL and seminar discussions in the earlier parts of the orthodontic curriculum11 and with Web-based presentations in dental radiology.12,13 We consider it unlikely that the positive responses to this teaching methodology were due mostly to a novelty effect.

Shifting responsibility of learning to the student by using self-instructional material has two advantages. First, it gives the faculty more flexibility in the allocation of teaching resources, so that faculty time can be used not only to transfer information but also to be sure that students understand the material. Second, it individualizes the learning environment for the students, allowing them to work with the teaching materials wherever and whenever they wish and to take as much time as they need in going through the material. At the predoctoral level, it has the potential to produce better-informed dentists and better interactions between other dental practitioners and orthodontic specialists.

We feel that the experience with this course demonstrates that the strategy of combining Web-based self-instruction and small-group discussions can be a viable way to augment faculty resources. The structured seminar outline and presentation of the same clinical examples in each seminar seemed successful in controlling variations among the small-group sessions.

This approach can become a way to share faculty resources among schools, so that an instructional expert in a particular area could manage the teaching of that subject at several institutions. For maximum effectiveness, the distant faculty member should be involved in both the development of the instructional modules and the small-group discussions. The rapid advances in video conferencing that are occurring now make it possible for a distant instructor to interact directly with small groups, either a group of seminar leaders for small-group discussions within large classes (as with dental students), or a small group of students (as at the graduate orthodontic level). At the graduate level, a distant seminar leader in an interactive video conference could potentially be as effective as one who was physically present—if, and almost surely only if, the clinical faculty at the recipient location supported this approach.

The development of these teaching modules was supported by the Orthodontic Fund, Dental Foundation of North Carolina, and the computer programming was done by Eric Coker of Retrorocket Multimedia, Durham, N.C. We thank Dr Wallace Hannum, School of Education, University of North Carolina, for consultation and advice and Ms Debora Price for statistical analysis and display of the data.