The aim of the study was twofold: (1) to compare the perceived educational environment at 2 points in time and (2) to longitudinally examine potential changes in perceptions of the educational environment over time.
The validated Dundee Ready Educational Environment Measure (DREEM), a 50-item, self-administered Likert-type inventory, was used in this prospective study. Employing convenience sampling, undergraduate chiropractic students were investigated at 2 points in time: 2009 (n = 124) and 2012 (n = 127). An analysis of 2 matching samples was performed on 27% (n = 34) of the respondents in 2009.
A total of 251 students (79%) completed the inventory, 83% (n = 124) in 2009 and 75% (n = 127) in 2012. The overall DREEM scores in both years were excellent: 156 (78%) and 153 (77%), respectively. The students' perceptions of teachers differed significantly between the 2 cohort years, decreasing from 77% to 73%. Three items received deprived scores: limited support for stressed students, authoritarian teachers, and an overemphasis on factual learning; the latter significantly decreased in 2012. In the longitudinal sample these items also displayed scores below the expected mean.
Students viewed the educational environment as excellent both in 2009 and 2012. The perceptions of teachers declined with time; however, this could be attributed to teachers' new roles. Certain aspects of the educational environment factored prominently during the comparative points in time, as well as longitudinally, and these ought to be further investigated and addressed to provide an enhanced educational environment.
INTRODUCTION
The last decade has witnessed a growing interest in the role of educational environments in health care professional training. The value of the educational environment for the quality of education is underpinned by research outcomes, and it is widely acknowledged that the environment is an important determinant of an effective curriculum and affects students' behavior and sense of well-being.1–3 Consequently, there is increasing recognition of the educational environment as an effective framework in student learning.4 Studies have shown that students' perceptions of the environment significantly influence educational results and academic advancement5–7 and that desirable learning outcomes are positively linked with aspects of sociopsychological environments.8,9 Moreover, dysfunctional environments are costly and counterproductive.10,11 Given the significance of the educational environment, it is important to examine its strengths and weaknesses and to determine how these can be modified to provide better learning experiences.
Empirical investigations of educational environments can be relatively complex, as they can encompass abundant layers and a multitude of different features, settings, and stakeholders. Students are among the key stakeholders, and their perceptions of their educational environment are highly useful sources of information; thus, educational environments can be partially characterized by interactions between different stakeholders. They embody many factors that can contribute to effective education, and they are the backbone in which a curriculum resides.12
The educational environment is an intricate web of emotional, intellectual, and physical strings, which are socially constructed by individuals.1,2,13 It constitutes a somewhat vague construct with a multitude of connotations, thus making definitions challenging—although there have been attempts.2 This difficulty is probably due to its all-embracing nature.13 It can be presumed that educational environments constitute not only externalized and tangible objective components but also subtle and internalized features, such as the “personality” traits of an institution. Thus, the educational environment is foremost a theoretical construct that cannot be measured directly; however, its pervasive and substantial effects are manifested in students' mundane experiences and perceptions, which can be explored and assessed.
Various instruments can be used to measure educational environments in health care professional education, each with strengths and drawbacks in terms of design, validity, and reliability. The Dundee Ready Educational Environment Measure (DREEM) is perhaps the most extensively used instrument.14 It has been used to explore, evaluate, and compare various aspects of undergraduate educational environments, such as institutional differences,15 levels of training,16–18 curriculum reform,19 and gender discrepancies.20,21 DREEM is suitable for gauging the environment in educational contexts,14 and it has been reported to have good psychometric properties in many diverse contexts, with evidence based on test content (content and construct validity)22–24 and internal consistency (reliability).19,25–28 The instrument has undoubtedly contributed to establishing a greater contextual understanding of professional health care education.
Although numerous studies have been carried out to explore medical, osteopathic, dentistry, physiotherapy, and nursing educational environments, there appears to be a gap in the scientific literature on analyses of chiropractic educational environments, with only a few studies endeavoring to identify students' perceptions.3,27,29
There is also a paucity of empirical investigations on changes in the educational environment over time. Edgren et al19 scrutinized the manner in which medical students perceived their educational environment and compared their DREEM findings from 2 specific time points during an episode of curricular reform. They found that the perception of the educational environment could remain high during ongoing curricular reform. Mojaddidi et al30 reassessed the perceptions of medical students using the DREEM inventory 3 years after their original study to identify changes in attitudes over time. They found problematic areas in the educational environment and suggested that these should be specifically targeted with remedial measures.
Importantly, there is a dearth of longitudinal pairwise scholarly investigations of the educational environment. Longitudinal studies are valuable for studying individual-level variation over time, in contrast to cross-sectional designs, which offer a snapshot of a population at a distinct point in time. Longitudinal studies investigating the educational environment are sparse, and to our knowledge, there are no existing longitudinal follow-up studies of students using pairwise comparisons that, in turn, employ the entire DREEM inventory.
In an earlier study,27 we reported cross-sectional data, but the paucity of comparative and longitudinal studies of the educational environment focusing on chiropractic undergraduate students motivated us to reinvestigate this group. There is empirical evidence that extrinsic modifications and remediation of the educational environment, as well as intrinsic vicissitudes such as students' time spent in a training institution, may alter perceptions of this environment.11,19,30 In this study, the overarching research question was “How do chiropractic students perceive their educational environment over a 3-year period?” An understanding of the temporal component of the environment—how it changes or is perceived to change over time—can help facilitate the development of educational environments that are apt for health care professional students. Thus, the aim of the study was twofold: (1) to compare the perceived educational environment at 2 points in time and (2) to longitudinally examine potential changes in perceptions of the educational environment over time.
METHODS
Empirical Setting
The study setting was the Scandinavian College of Chiropractic (SCC), Sweden. The SCC is a university college offering a 5-year, full-time undergraduate academic program in chiropractic and is the only state-recognized chiropractic educational institution in Sweden. The program has a spiral curricular structure. It is divided into a conventional preclinical phase, with theoretical and practical training in traditional and formal classroom settings, and a clinical phase at the institution's outpatient clinic. After graduation and a 1-year internship in public health care, the National Board of Health and Welfare issues a professional status qualification in chiropractic (registered chiropractor).
Study Design and Methodology
In the present study, we implemented a prospective comparative and longitudinal quantitative descriptive survey design using the DREEM inventory. The study was part of a larger prospective research project employing a mixed-method multiple–case-study methodology anchored in a pragmatic research tradition.
Participants and Sampling
A nonprobability convenience sample of volunteer undergraduate chiropractic students from 10 batches was invited to participate in the survey: years 1 to 5 in the 2009 and 2012 cohorts. Undergraduate students adhering to an individually tailored curriculum were excluded. The DREEM inventory was administered by a member of staff during classes to ensure a high response rate. Students not present at the time of data collection were invited to participate via e-mail.
Data Collection
DREEM is a self-administered, closed-ended inventory relating to a variety of topics of direct relevance to educational environments. Translated and validated for use in Sweden,19 it comprises 50 statements scored from 0 to 4 using a 5-point Likert response: 0 = strongly disagree, 1 = disagree, 2 = unsure, 3 = agree, and 4 = strongly agree. The items are congregated into 5 subscales covering different features of the educational environment: students' perceptions of learning (SPL; 12 items/maximum score 48), students' perceptions of teaching (SPT; 11 items/maximum score 44), students' academic self-perceptions (SASP; 8 items/maximum score 32), students' perceptions of the atmosphere (SPA; 12 items/maximum score 48), and students' social self-perceptions (SSSP; 7 items/maximum score 28). Nine of the items are negative statements and are therefore scored in reverse. Thus, for all the item and subscale scores, a higher score designates a more positive response. The items can be analyzed on 3 levels: individually, pooled into 5 subscales, and overall. The instrument has an overall score of 200, and overall and subscale scores can be interpreted against standard guidelines.31 Edgren et al19 highlighted the importance of analyzing data on an item level because the overall perception of the educational environment and the subscales could conceivably conceal explicit educational problems. Individual item scores can also be examined to pinpoint definitive strengths and shortcomings.31 Items with a mean score greater than 3.5 mainly represent strong areas; a score of less than or equal to the expected mean of 2 indicates problematic areas; and scores between 2 and 3 indicate areas in which improvements could be made.
Data Analysis
The completed inventories were manually entered, using a double-entry procedure, into the Statistical Package for the Social Sciences (SPSS) version 20.00 (IBM Corporation, Armonk, NY) for descriptive and inferential statistical analysis. As 9 of the 50 items (18%) from the instrument are negatively stated, corrections were made; thus higher scores designated disagreement with these items. The items were analyzed on 3 levels—individually, pooled, and overall—only if all items were completed by the respondents. The main focus of the analysis was the individual item level. The data from all levels were reported as averages through means. The overall and subscale scores were expressed as percentages of the respective maximal attainable scores.22,32 The data distribution was assessed visually via boxplots by contrasting potential discrepancies among the parameters of central tendency, evaluating the skewness and kurtosis of the distributions, and employing Kolmogorov-Smirnov and Shapiro-Wilk tests.
The criterion variables were the perceptions of the educational environment as measured by the overall, subscale, and individual scores of the inventory, and the predictor variables were the cohorts' year (2009 and 2012) and year of study (1 to 5). Following the guidelines outlined by Swift et al,33 the 5 response categories were trichotomized into agree/strongly agree, unsure, and disagree/strongly disagree. Cronbach α was employed to assess the internal consistency of the subscale scores of the instrument, and a minimum coefficient α of .70 was employed to indicate an adequate level of consistency.34
Nonparametric statistical tests were performed and designated to evade influences of the distribution of the data. To compare 2 independent samples, the differences were calculated using the Wilcoxon-Mann-Whitney test. The χ2 test was used to test the proportions of clusters (agree/strongly agree, unsure, and disagree/strongly disagree) if the observed proportions were 5% or more in both clusters. To compare 2 matched samples, the Wilcoxon signed rank test was employed. For the proportions of clusters in the matched samples, an type of analysis akin to that stated above was performed, replacing the χ2 test with the McNemar test of equal proportions. For both the 2 independent and 2 matched samples groups, comparisons were made for each individual item.
Regarding the overall and subscale scores, the main focus of the analysis was to compare DREEM scores between cohorts' year and year of study. As there is no nonparametric equivalence to a 2-way analysis of variance, we employed the Wilcoxon-Mann-Whitney test for independence between group analyses and the Wilcoxon signed rank test for the matched-samples analyses. These tests were repeated 5 times. Due to multiple comparisons (5 groups), a reduced p value of .01 was used by employing the Bonferroni adjustment of primary endpoints to control for the risk of mass significance.35 In all other circumstances, probability values of less than .05 were considered statistically significant.
Effect size was calculated by dividing the mean of the change scores by the pooled standard deviation.36 Effect size statistics <0.2 indicated a small meaningful magnitude of change, 0.2–0.7 a moderate change, and >0.7 a large change. These analyses were performed for all 3 analytical levels.
Ethical Considerations
The Regional Ethical Review Board in Stockholm approved the study (2012/416-31/5). Participation was voluntary, and the participants were informed about the study both orally and in writing. Written informed consent was obtained from the participants prior to completing the DREEM inventory and the interviews. All collected data were anonymized to maintain the integrity of the individual responders, and the data were handled and stored in accordance with the tenets of the World Medical Association Declaration of Helsinki (2008).
RESULTS
In a total population of 318 students from 10 batches, 251 undergraduates (79%) completed the inventory. In 2009, 124 of 149 students (83%) completed the questionnaire, and no questionnaire was rejected as a result of incompleteness. In 2012, 127 of 169 students (75%) completed the questionnaire, and 3 questionnaires were rejected due to incomprehensiveness. Fourteen percent (n = 34) of the students were available for follow-up, constituting 27% of the 2009 sample, and their scores from years 1 and 2 in 2009 were compared with those from years 4 and 5 in 2012. Table 1 shows the demographic characteristics of the participants.
The internal consistency of DREEM, conducted using the subscale scores, showed Cronbach α values for SPL, SPT, SASP, SPA, and SSSP of .781, .725, .512, .759, and .743, respectively, in 2009; and .710, .645, .621, .727, and .569, respectively, in 2012.
Overall and Subscale DREEM Scores at 2 Points in Time
Summary data on the overall DREEM and subscale scores are presented in Table 2. The overall mean score was 156 (78%) in 2009 and 153 (77%) in 2012. The difference between the 2 means was not statistically significant. In 2009, the highest overall mean score (84%) was in year 1, and the lowest overall mean score (75%) was in year 4. Similarly, 3 years later the highest score (78%) was in year 1, and the lowest score (75%) was in year 3. In 2009, year 1 students also had higher overall scores compared to their counterparts in 2012, but when the p values were adjusted for multiple comparisons, this was not significantly different. The subscale scores are displayed in Table 2, together with the percentages of maximum score. In the analysis and comparison of the 2009 and 2012 cohorts, only SPT showed a statistically significant difference (p = .006). With regard to the SPT subscale, there was a statistically significant difference (p = .002) between year 1 students at both points in time.
Longitudinal Overall and Subscale DREEM Scores
The overall mean scores for the longitudinally surveyed sample (n = 34) was 156 (78%) in 2009 and 158 (79%) in 2012. The difference between the 2 means was not statistically significant. Figure 1 shows the distribution of the subscales—which does not display any significant differences—between the 2 points in time.
Comparison of Individual DREEM Scores at 2 Points in Time
Table 3 presents the scores for the individual items. The total item mean amounted to 3.1 both in 2009 and 2012. In 2009, the total highest mean score (3.7) was reported for the individual items “The teaching helps to develop my competence” and “I feel comfortable in class socially.” The lowest mean score (1.8) was reported for the items “There is a good support system for students who get stressed” and “The teachers are authoritarian.” In 2012 the highest mean score (3.7) was also reported for the item “The teaching helps to develop my competence,” and the lowest was 1.5 for the item “The teaching overemphasizes factual learning.”
In 2009, 4 items scored below the expected mean. In 2012, 3 of these items scored likewise. Regarding the item “There is a good support system for students who get stressed,” respondents in both cohorts displayed low proportions of agreement and high proportions of disagreement and uncertainty. The proportion of disagreement with the negatively stated item “The teachers are authoritarian” was low in both 2009 and 2012, while the proportion of agreement was high in both 2009 and 2012. Concerning the negative item “The teaching overemphasizes factual learning,” there was a statistically significant (p < .001) reduction in the mean between the cohorts. There was also a low proportion of disagreement and a high proportion of agreement with this item, and the findings were statistically different for both years. In 2009, the score for the item “This school is well timetabled” was below the expected mean. However, in 2012, the mean surpassed the threshold, and this difference was statistically significant (p = .009). Moreover, proportional disagreement with this statement was more than 20%. The mean values for the statement “The enjoyment outweighs the stress of studying chiropractic” were statistically significantly higher (p = .011) in 2012 than in 2009. The mean values of the 4 items “The teachers are good at providing feedback to students,” “The atmosphere is relaxed during seminars/tutorials,” “The teaching is too teacher centered,” and “The students irritate the teachers” were significantly lower in 2012 (p = .007; p = .005; p < .001; p < .001, respectively), and there were statistically significant proportional differences in the response pattern.
Comparisons of Individual DREEM Scores of Longitudinal Matching Samples
Among the 34 follow-up participants, the total item mean scores were 3.1 and 3.2 in 2009 and 2012, respectively. Table 4 displays the individual DREEM items of the 2 matching samples. In 2009, the total highest mean score (3.7) was reported for the individual items “I have good friends in this school,” “I feel comfortable in class socially,” and “The atmosphere is relaxed during lectures.” The lowest score (1.5) was recorded for the item “There is a good support system for students who get stressed.” Four items scored less than or equal to the expected mean, while 7 items (14%) scored greater than 3.5. In 2012, the highest mean score (3.8) was reported for the item “The teaching helps to develop my competence.” The lowest was 2.0 for the items “There is a good support system for students who get stressed” and “The teachers are authoritarian.” Two items (4%) scored less than or equal to the expected mean, while 8 (16%) scored above 3.5.
Two items were below or at the expected mean both in 2009 and 2012: “There is a good support system for students who get stressed” and “The teachers are authoritarian.” However, the mean score for both items was higher in 2012, though the score was only statistically significant (p = .046) for the former. More than 20% of the respondents disagreed with the statement “There is a good support system for students who get stressed,” but the proportion of disagreement was significantly lower in 2012. A high proportion reported uncertainty, and a low proportion agreed or strongly agreed with the statement. There were low proportions of disagreement and high proportions of agreement with the negative statement “The teachers are authoritarian.” Two items, “This school is well timetabled” and “The teaching overemphasizes factual learning,” which were below the expected mean in 2009, exceeded the threshold in 2012, but the difference was not statistically significant. There were high proportions of disagreement with the first statement and low proportions of disagreement and high proportions of uncertainty and agreement with the second statement.
DISCUSSION
Key Findings
The aims of this investigation were (1) to compare the perceived educational environment at 2 different points in time and (2) to longitudinally examine potential changes in the perceptions of the educational environment over time. The overall inventory response rate was 79%: 83% responded in 2009 and 75% in 2012. This can be considered satisfactory and may reflect students' interest in completing the study. The overall DREEM scores in 2009 and 2012 were very high: 156 out of 200 (78%) and 153 (77%), respectively. In 2012, the score dropped slightly, but it remained within the range of an “excellent” educational environment.31 Only the SPT subscale differed significantly between the 2 cohort years, decreasing from 77% to 73%. The scores for 3 items received deprived scores over time: limited support for stressed students, authoritarian teachers, and an overemphasis on factual learning, with the third significantly lower in 2012. In the longitudinally surveyed sample, the same 3 items recorded the lowest scores and scored below the expected mean. It is plausible that the positive perception of the educational environment increased with time when the students were surveyed longitudinally.
The Study's Contribution and Relation to Empirical Evidence
Internal Consistency
Our data demonstrated that the Cronbach α values for the DREEM subscales varied, with 4 of the 5 subscales surpassing the α threshold34 in 2009 but only 2 doing so in 2012, indicating that their internal consistency was compromised. Yusoff et al37 asserted that this may jeopardize the ability of the DREEM inventory to give analogous results if a similar population is being studied at different times and occasions. Previous research has recently pointed to discrepancies in the DREEM in relation to the variability of Cronbach α in the 5 subscales.38–40 Thus, it has been suggested in the literature that low α values could be a result of an inadequate number of statements, poor interrelatedness between items, or heterogeneous constructs. Conversely, very high values may indicate that some items are redundant because they are testing the same construct, though in a diverse form.41,42 Our data on low α values might indicate that there are more than 5 constructs being measured by the DREEM, a notion which has been supported by others.43 Such multidimensionality could lead to Cronbach α underestimating the level of reliability.44
Overall and Pooled DREEM Scores
The overall DREEM scores in the present study are, as far as we know, the highest reported in the published literature. The closest published results to ours are from the United Kingdom,21,45 Australia,46,47 and Sweden.19 The reason for the high score can only be speculated upon, but it would be intriguing to explore whether small private institutions score predominantly well. This notion has recently been supported, with investigators suggesting that higher overall and subscale DREEM scores could be attributed to smaller class sizes in private institutions.48 However, scholars have shown that even in small training institutions with a perceivably good educational environment, as measured with the DREEM inventory, subtle but important gender-, ethnic-, and minority-related issues could overshadow the propensity of a scanty educational environment.49 Furthermore, McKendree45 proposed that a “new school” in which stakeholder enthusiasm pervades the entire institution could partly explain high perceptions of the educational environment. The term “organizational saga”50 has been used to explain why some institutions in higher education are regarded as very good. Burton50 pointed out that a saga often begins with the strong vision of a small group with a mission, and that this phenomenon is often more tangible and concrete in smaller organizations.
Contrasting the subscale scores with those proposed by the developers22 revealed that they were in the upper part of the range. The following commonalities between the 2 cohorts were also revealed: the students considered the teaching to be of high quality; they felt that teachers were striving to do their best; the students showed academic confidence; and there was a good feeling overall among the students about the educational atmosphere and their social situation. The only subscale that differed significantly, with a moderate effect size, between the 2 cohort years was the perceptions of teachers. This might be due to a recent curricular modification by the SCC institution, with the curriculum changing from the traditional student-centered model to a problem-based, student-active spiral curriculum with less emphasis on traditional lecturing. During this educational shift, it is possible that the teachers took time to find their feet in their new roles as facilitators rather than traditional lecturers.51 Additionally, our analysis revealed that in 2012, year 1 students were significantly more negative, exhibiting a large effect size, in their perceptions of teachers than in 2009, which may be reflective of the curricular shift and the teachers' new roles. Moreover, the analysis of individual items showed a significant reduction in perceptions of the quality of feedback, teachers becoming irritated during teaching, teacher centeredness, and the atmosphere during seminars. Similar findings have been reported by others19,30,52 and might be regarded as implicit evidence that teachers are inclined toward traditional ways of teaching and are uncomfortable with their new roles.
In accordance with other studies, this reassessment corroborates the finding that the perception of the educational environment seems to be at its highest in the first and last years of an educational program, with a measurable slope in the middle (in our case, years 3 and 4).3,27,53–55 However, in 2009, 1st-year students were more positive about the educational environment than their peers in 2012; though the Bonferroni adjustment for mass significance showed that this was not significant, the same sloped shape was apparent halfway through the program. In congruence, researchers have proposed that the initial enthusiasm of many students appears to decrease during the course of their studies and that the perceived deterioration of the educational environment is not entirely due to educational delivery but also to individual factors, such as becoming older, more critical, and autonomous.21,56 Young students' happiness in relation to entering adult life and taking up higher education studies could explain the initial high scores, and it seems intuitively plausible that the scores would rise again as students approach the culmination of a long educational program. In contrast, in a recent comparative study over time, Mojaddidi et al30 reported that the scores of clinical-phase students regarding their perception of the educational environment were higher than those of preclinical-phase students. They argued that this was most likely an indication of change in the educational environment over time. Likewise, Kang et al57 recently reported that this decline in the DREEM scores halfway through a training program could be explained by the commencement of clinical education and the differences in perceptions of the formal vs the clinical environment. However, there is empirical evidence suggesting that students who start a training program with a higher expectation of their educational learning environment are prone to having more positive overall perceptions of the environment throughout the course of the program.57
Individual Item DREEM Scores
Based on the individual item scores surpassing the estimated mean and the demarcated cutoff of 3.5, which, as per the DREEM constructors,31 represent strong areas, students perceived at both points in time that teachers were knowledgeable and devoted to developing the students' competencies, and that the training seemed relevant for their professional career. There was also a perception of good camaraderie and belonging to a good social situation.
As pointed out by Edgren et al,19 scores from overall and subscale perceptions of the educational environment may conceivably veil the presence of explicit educational problems; therefore, particular attention must be paid to interpreting the DREEM inventory on an item level. As such, the 3 items persistently receiving deprived scores over time are discussed below as separate entities.
Limited Support for Stressed Students
In agreement with other studies, our data showed dissatisfaction and poor scores in the perception of support systems for stressed students.18,19,30,57 Edgren et al19 acknowledged that in the majority of published DREEM study results, this item tends to score low and seems to be a communal problem in professional health care training. Professional health care education can be demanding and can produce diverse stressors.58 Such stress has been shown to affect well-being and hamper academic performance.59 The SCC has adopted a series of measures aimed at dealing with students who exhibit stress. This was, in some ways, evident in the significant increase in 2012 of the item concerning the perception that enjoyment overshadows the stress of studying, although the effect size was only moderate. Further remedial actions that could be taken include increasing leisure time activities, improving interactions between students and faculty, and providing peer counseling and advisory services.60 Mojaddi et al30 stipulated that stress support and academic affairs counseling can play a significant role in increasing the perception of the educational environment. An effective support system is necessary to provide help during a likely stressful health care professional education. The faculty can help to ensure a safe and respectful environment in which students are invited to pose questions and clarify concepts. Mentoring can play a role in providing emotional support, advice, and encouragement.61 Kahlen et al62 showed that mentorship can provide psychosocial support and facilitate students' professional identity and personal development.
Overemphasizing Factual Learning
Many DREEM studies have highlighted students' perception that teaching overemphasizes factual learning,3,18,19,47,54,57,63 thus insinuating that students are employing surface-learning approaches to the detriment of deeper learning.64 Perceptions of factual overload have been shown to be correlated with surface-learning approaches.65 Indeed, many features of health care professional training might propel students toward surface learning.66 Furthermore, scholars have shown that students who perceive the educational environment deleteriously may be those who are less academically proficient.67,68 Although many institutional efforts have been made at SCC to combat factual overload, it is evident from the current study that there is further room for improvement, as the item mean values were below the threshold both in 2009 and 2012 and statistically significantly lower in 2012. However, in the longitudinal data, we detected an increase in this perception, though not significant, with the item demonstrating high proportions of uncertainty and agreement of the statement.
The learning of facts is not ambiguous as such. However, empirical evidence suggests that learning is less optimal when facts are contextually distanced from where they serve purpose and meaning.69,70 It is feasible that the perception of overemphasizing facts is mostly derived from the biomedical sciences during the preclinical phases of health care professional training. Whitehead71 asserted that the biomedical sciences in professional health care education need to be conceptualized not as a set of facts but simply as one of numerous essential forms of knowledge in education. She further proposed that alternative forms of knowledge derived from other areas, such as the social sciences and humanities, must be seen as equally important and must be fully incorporated into the educational system.71 Research has shown that educational environments that enhance authentic contexts facilitate meaning making, stimulate the application of knowledge, and promote iterative reflection, thus combating both teacher centeredness and the emphasis on factual learning.19,72–74
Authoritarian Teachers
Despite a paradigmatic shift in health care professional education toward student centeredness and self-directed, lifelong learning, as well as the transformation of teachers' roles from the “sage on stage” to the “guide by the side,”75 similar to many other studies3,26,54,57,76 we found an overall perception among students that teachers were authoritarian. Also, significant reductions in some items pertinent to teachers, teaching, and the atmosphere in our data may further imply that teachers are perhaps prone to traditional styles of teaching based on teacher-centered attitudes and practices. Significant reduction in the score for the statement “The students irritate the teachers” during the 2 points in time may also be regarded as supportive evidence of the perception of autocratic teachers. Scholars have underscored that teachers should be trained to improve their teaching skills, both in clinical and formal settings, to learn the skill of providing constructive and purposeful feedback.77–80 Haden et al80 stated that when both faculty and students demonstrate humanistic values, learning without coercion can occur. However, in an ensuing qualitative interview with students (data not reported here), the participants had difficulty explaining the connotation of the word “authoritarian,” thereby making this item a possible instrumental artifact, which in turn, raises questions over its face validity. While we reported on this earlier,54 we believe that there may also be cultural differences in the connotation of the word authoritarian. It is possible, contrary to previous claims made by the DREEM developers,22 that the inventory might not be independent of culture, a viewpoint supported by others.24,40,81 On the contrary, both in 2009 and 2012, our data revealed that students demonstrated disagreement with the statement “The teachers get angry in class,” displaying scores above the estimated mean and greater than the cutoff for strong areas. Nevertheless, the notion of authoritarianism could form an accurate assessment of the students' perception and must be investigated further.
Longitudinally Examined DREEM Changes
Our longitudinal data did not display any statistically significant changes regarding the overall or subscale scores. This is contradictory to the conclusions drawn by Shankar et al,82 who detected such changes in a follow-up study. However, based on our longitudinal findings and on an item-level analysis, we maintain that students' perception of the educational environment becomes more positive with time. First, even though the overall or subscale scores did not differ significantly, 62% of the items increased with time while 12% remained the same. Second, the items showing statistically significant changes, with moderate effect sizes, were all in the positive direction, indicating improved perceptions of the environment. Third, 3 items indicating that students were significantly more positive about the educational environment were pertinent to the perception of teachers and teaching, ostensibly an indication of change in the educational environment over time. Fourth, the 3 aforementioned items receiving deprived scores in the cohort analysis at both points in time displayed positive mean changes, although only 1 was statistically different. The data also indicated positive proportional alterations in the trichotomized response patterns. Lastly, in 2009, 4 items were equal to or below the expected mean, but in 2012, only 2 items were equal to the expected mean and none scored below. Still, we concur with Edgren et al19 and Mojaddidi et al30 and cannot completely rule out cohort effects as an explanation, and neither can we dismiss discrepancies between preclinical and clinical environments. However, many scholarly investigations using DREEM have shown that perceptions of the environment decrease with the time spent in the training institution,3,27,29,53–55 except for slight score escalations in the final phase. Importantly, however, most of these studies have employed cross-sectional—not longitudinal—designs.
Synthesizing a Model
Using an abductive reasoning approach based on the aforementioned findings, empirical evidence, and existing theoretical frameworks, we propose a model that could illustrate and depict the reciprocal connections between educational environment issues and negative perceptions (Fig. 2). This iterative cycle should be interrupted by creating a congenial environment wherein educational processes become an encouragement rather than a stressor for both students and teachers, result in the establishment of adequate support systems for handling stress, extricate teachers from traditional teaching mind-sets, and construct student-centered environments wherein teaching/learning processes promote meaningful factual learning that is contextualized and decisive.
Methodological Reflections and Limitations
There are some limitations regarding the study method. Although the comparative samples were of a reasonable size, the longitudinal sample was quite small; therefore, it was difficult to draw conclusive results. Notwithstanding, the results are congruent with our data comparing 2 independent samples. Despite some methodological drawbacks, therefore, it was relevant to investigate our longitudinally derived data to explore the educational stakeholders (undergraduate chiropractic students) and their evolvement and adaption within the educational environment over time.
We did not test demographic variables such as sex, age, and cultural background, which could be regarded as a drawback. However, the purpose of the study was to investigate the training institution as a whole, and we did report on these characteristics earlier.27,49
There is controversy regarding how Likert data should be analyzed—a topic debated in the scientific literature for nearly 80 years83–85 —and the DREEM inventory has also been subjected to this discourse.33,86 The rationale behind our choice of nonparametric analysis was grounded in the ordinal nature and non-Gaussian distributions of data. However, there is evidence on the suitability of parametric techniques.84,85,87 One could argue that the sums of the independent items, constituting subscales, are likely to be less skewed and more normally distributed than the items themselves, and that treating the data from subscales as ordinal in character averts the use of more potent modes of analysis, such as parametric statistical inferences. Nevertheless, theoretically, only continuous variables can be normally distributed, while a categorical variable with 5 levels (such as Likert items) cannot. The variable can be symmetric, thus permitting recoded values to be interpreted with parametric tests. However, whether this is appropriate or not has been a long-standing issue. Pragmatically, as a type of sensitivity analysis, we also employed these parametric methods and performed a post hoc analysis (not reported), but we did not detect any major dissimilar results, except for diametric significance levels for less than 9% of the items, which could be attributed to differences between means and proportions.
Nonparametric statistical analyses are calculated using the median as a measure of central tendency. However, concurring with Swift et al,33 we believe it is more meaningful to present means rather than medians because the median can only take 1 of the 5 possible scores. For skewed distributions, which seem rather common for DREEM items, an item with a satisfactory central measure may still conceal a high proportion of negative responses. We therefore adhered to the recommendations outlined by Swift et al33 and trichotomized Likert response categories and reported the means.
It has also recently been proposed that values below the expected mean, the threshold as recommended by the developers, could be a priori elevated to 2.5 to attain a more stringent analysis.33 However, in the current study, we chose to follow the guidelines of the originators22 and employed 2.0 as the limit.
Despite the large number of studies utilizing DREEM to explore students' perceptions of their higher education, very few psychometric reports have been published. Consequently, some recent concerns have surfaced about the psychometric robustness of the DREEM instrument.24,38–40 Although the DREEM instrument was initially reported to have good construct validity in its original context,22,23 more recently, investigators have questioned the stability and construct validity of the measure in other contexts.37,38,40 Reproductions of the original scale structures have been only moderately successful, probably indicating weaknesses in the instrument.24,39 Because reliability and validity are products of data gathered,88 there are also some questions regarding the cultural validity of data derived from the DREEM instrument used within our study. Vaughan et al40 recently proposed a new short version of the DREEM instrument based on data from an osteopathic setting, but this warrants further investigation, as does the originally proposed subscale structure. These authors40 also advocated caution when calculating the overall sum score on the basis of psychometric results, as the instrument was unable to gauge a single underlying construct. Still, it is customary in educational environment research to report overall DREEM sum scores for cross-institutional comparisons, and until this issue has been further psychometrically explored, we shall report these results.
It has been postulated that high α coefficients could reflect redundancy among items89,90 and multidimensional scales.91 In the present study, we did not dispatch any overall α scores, though those scores seemed rather high (not reported) and might suggest that the items correlated strongly with each other.42 Cronbach α is grounded in the “tau equivalent model,” which assumes that each test item measures the same latent trait on the same scale.92 It is customary among many authors utilizing the DREEM inventory to report on the reliability of the overall α score derived from all the DREEM items. However, this can be inappropriate, as it violates the assumption upon which the model is constructed—which can give an inflated score.
Jakobsson et al24 performed a psychometric investigation using the Swedish version of DREEM on a sample of medical students and reported that it was valid and reliable, except for the subscale structure; they proposed a new 5-factor solution. However, they subsequently stated that the results obtained with the new factor design were not superior to those of the original. Despite the need for continued research to explore and determine the psychometric properties of the DREEM instrument in a variety of contexts and settings, it has contributed immensely to a greater contextual understanding of the educational environment.
The use of mono-method approaches and the employment of survey-based research can be problematic and dubious. Quantitative instruments merely create an instantaneous representation of how different stakeholders perceive their educational environment but cannot offer rich data regarding the stakeholders' experiences of these environments and the concerns underlying deprived scores; neither can they illuminate other constructs that are not encompassed by the inventories. To gain a deeper understanding of the multilayered phenomenon of the educational environment, in-depth qualitative investigations could shed light on the complex characteristics and enhance our understanding of both the strengths and weaknesses of the current context, as well as of more eclectic factors influencing educational stakeholders' experiences and perceptions of their environment.
Relevance for Future Health Care Education Research
Empirical and theoretical evidence suggests that although the concept of the educational environment is rather intangible and poorly understood, its effects are extensive, tangible, and persuasive. Concurring with others,40 we argue that this phenomenon is not only due to the perceptions of marginalized individuals but also because of multidimensional factors with noticeable effects on educational outcomes. Thus, there is a need for further research on the factors and concepts involved, including definitions of the concept of the educational environment and its constituents.
Although newly emerging psychometric evidence from a multitude of heterogeneous contexts may cast doubt on the psychometric properties and possible multicollinearity of the inventory, it is still the mostly widely used instrument for assessing the undergraduate professional health care educational environment. However, more empirical work should be performed to address issues of dimensionality regarding the DREEM subscales. Individual DREEM items that consistently score below the expected mean in conjunction with diverse proportional response patterns should be explored psychometrically and qualitatively with regard to item construction, syntax, and phraseology. Because the DREEM was developed about 15 years ago, the moment may be opportune to revise the items and possibly incorporate new features, such as perceptions of physical and technological aspects of the educational environment.
It has been noted that validity does not pertain to the instrument as such but is rather a characteristic of the interpretation of the generated results.88 Richardson93 pointed out that cultural discrepancies, fluctuations in student characteristics, and diverse and changeable modes of teaching require iterated scrutinizing and fine-tuning of instruments. Hence, further research should continue to explore, strengthen, and develop the psychometric properties of the instrument to develop a sound version, or even construct an entirely new instrument, as there seems to be a need for one.
Only a few instruments have been developed to specifically assess students' perceptions of the clinical educational environment, such as the Undergraduate Clinical Education Environment Measure (UCEEM),94 the Dutch Residency Educational Climate Test (D-RECT),95 and the Postgraduate Hospital Educational Environment Measure (PHEEM).96 Because the DREEM is perhaps not completely appropriate for measuring clinical environments, such environments might be investigated using more suitable instruments in order to equally and unambiguously cover all aspects of the educational environment.
Studies using quantitative approaches emphasize the measurable attributes of a phenomenon. However, the use of instruments such as DREEM to assess perceptions of an educational environment can be complex and arduous because of the possibility of excluding certain fundamental and explicit factors. In qualitative approaches, research attempts to understand the meaning, breadth, and variations of a phenomenon. Employing qualitative methods, such as focus groups, semistructured interviews, or ethnographic approaches, and collecting data from key stakeholders and/or outliers could be valuable in further exploring and understanding the concept of the educational environment. It is our belief that rigorous, comprehensive, and in-depth qualitative investigations are justified when exploring additional aspects of the educational environment that are not captured instrumentally, such as overall experiences, emotional aspects, features of reciprocated dependencies among stakeholders, issues of inequity, and the hierarchical traits of the organization.
Students are one of the key stakeholders, and their perceptions of their educational environment are a pertinent source of information. However, as asserted earlier, the perception of the educational environment among student cohorts is idiosyncratic and may differ widely on a year-to-year basis.54 Notwithstanding, students are only one side of the coin; the perceptions of faculty and other stakeholders are equally important. As teachers often remain in an educational environment for extended periods, it is plausible that they would perceive the environment more consistently. Scholars have drawn attention to the paucity of empirical studies investigating teachers' perspectives on the environment.1,2,56,97 There is a need for further research on how teachers perceive and experience the environment, as they are an intricate part of the environment perceived by students.
CONCLUSION
The undergraduate chiropractic students in this study perceived the overall educational environment to be excellent in both 2009 and 2012. The results further indicate that there were signs of improvement in students' perceptions of the educational environment over time. The students' perceptions of teachers declined with time, which could be attributed to teachers' new roles following the institutional curricular changes. Perceptions relating to limited support for stressed students, teachers being authoritarian, and the overemphasis on factual learning scored low but were consistent over time and may thus be important aspects to investigate further and consider in impending efforts to improve the overall excellent results.
ACKNOWLEDGMENTS
The authors wish to thank Eva Hagel for statistical consultation as well as the chiropractic students at the Scandinavian College of Chiropractic who dedicated their time to participate in the study.
FUNDING AND CONFLICT OF INTEREST
There were no external sources of funding and no conflict of interest that may bias the outcomes of this study.
REFERENCES
Author notes
Concept development: PP, TS, KBL. Design: PP, TS, KBL. Supervision: PP, TS, KBL. Data collection/processing: PP. Analysis/interpretation: PP, TS, KBL. Literature search: PP, TS, KBL. Writing: PP, TS, KBL. Critical review: PP, TS, KBL.