We outline the framework of a collaborative process to redesign an existing 5-year health education program, which may prove useful to other similar institutions. The aim was to strengthen evidence-based practice and curriculum alignment.
A whole-of-program approach was used to restructure the existing courses into 3 “streams”: professional practice, clinical research, and clinical science. The process incorporated a series of facilitated workshops organized by the department director of learning and teaching and the faculty facilitation team, and it was inclusive of all available members of the department, a clinic supervisor, a sessional (casual teaching) staff member, and a recent graduate of the program.
Unit content and assessments were restructured to progress the program learning outcomes from year to year. The undergraduate program was redesigned to create a more logical learning pathway for students. Consolidation of subject topics in the postgraduate program allowed for the development of stand-alone research-only units.
The mechanism of curriculum mapping allowed for discussion about the flow of information from year to year and how evidenced knowledge and understanding can be developed. It is necessary that everyone participates and understands the importance of program goals as developed by the process. Because drift in curriculum can occur incrementally over the years, to be effective, the program requires ongoing monitoring and regular collaboration to continue improvements.
INTRODUCTION
The Institute of Medicine (IOM) Committee on the Health Professions Education Summit in 2002 recommended that all health care professionals and trainees practice evidence-based medicine.1 Evidence-based practice (EBP) has been incorporated into health education worldwide because clinical research evidence is increasingly seen as a vital element of clinical decision making.2–4 The “driving force” behind evidence-based medicine since the 1980s has been David Sackett5 at McMaster University in Canada. In his words, evidence-based medicine integrates “individual clinical expertise with the best available external clinical evidence from systematic research.”6 In this approach, the research is not considered alone but in combination with the experience and expertise of the clinician and the patient's values.7,8 Evidence-based health education must therefore develop critical research skills, good clinical practice based on solid clinical science, and a patient-centered approach.
Evidence-based medicine was introduced into medical schools in Canada and the United States in the late 1980s9 and is incorporated into their accreditation standards.10 A review of publications from 2006 to 2011 found that EBP is found in most medical schools around the world, but it varies a great deal across institutions.11 A survey of 20 medical schools in the United Kingdom similarly showed that it is taught in most schools, but there is considerable variation, and limited time is often cited as a problem.12 Integrating change has been challenging.13 A survey was published in 2014 on EBP training in Canada and the United States, and from the 115 medical schools that responded, lack of time in the curriculum, difficulty integrating with clinical care, lack of faculty interest, and lack of staff knowledge were listed as some of the barriers.10
An audit of Australian accreditation documents for 12 health professions (medicine, nursing, midwifery, pharmacy, physiotherapy, dentistry, psychology, occupational therapy, optometry, podiatry, osteopathy, and chiropractic) was published in 2016.14 It found that only physiotherapy and psychology fully complied with the 2005 Sicily statement on the 5-step model of training in evidence-based practice; medicine and optometry complied with 4 of the statements, pharmacy with 3, and chiropractic, osteopathy and podiatry with 2.
Chiropractic has lagged on the uptake of EBP. In a 2010 editorial tracking 100 years of chiropractic history since the influential Flexner Report, it was concluded that “there are few efforts in chiropractic education…to revisit the core curriculum and ascertain where there is need for improvement based on the needs of the 21st century.”15 Changes have been made in some countries and some institutions. For example, in Denmark, chiropractic education is run by the Institute for Sports Science and Clinical Biomechanics in the Faculty of Health Sciences at the University of Southern Denmark, and here chiropractic and medical students study together in the same evidence-based bachelor's program.16 In another example, the University of Western States, Portland, Oregon, partnered with Oregon Health & Science University to integrate EBP knowledge, skills, and behavior into their chiropractic curriculum.17
From the perspective of chiropractors in clinical practice, a US national survey was done in 2012–2013 which captured the attitudes toward EBP from a representative sample of 1314 respondents predominantly in practice for more than 10 years (60%). They reported generally favorable attitudes to, but less use of, EBP, and only 17% indicated that EBP coursework was a large part of their education. Almost 90% wanted to improve their EBP skills.18 An Australian national survey of registered chiropractors, published in 2013, yielded a 13% response rate (584). It came to the same conclusions: EBP was well favored and many wanted to improve their knowledge, but many did not use clinical practice guidelines, an indication of the use of EBP.19
Innes et al20 recently systematically audited how comprehensively EBP is represented in Councils on Chiropractic Education, the regulatory and licensing bodies for chiropractic education. They found that there was a slow uptake of EBP terminology and trends. In Australia, the Council on Chiropractic Education Australasia (CCEA) has stated requirements in the areas of research and evidence-based teaching (section 4.4.2): “The principles of scientific method and evidence-based health care, including analytical and critical thinking, must be taught throughout the curriculum. The curriculum may include elements (such as elective research projects) for training students in scientific thinking and research methods.”21
The need for evidence-based chiropractic education has been well articulated in the profession. In a 2015 keynote speech delivered to the Chiropractic & Osteopathic College of Australasia, Bruce Walker, head of discipline for chiropractic and associate dean of research in the School of Health Professions at Murdoch University in Australia, gave as one of the keys to fully legitimize the profession that content be taught “in the context of the evidence and that students obtain the necessary training to question and critically appraise.”22
We identified that a concerted and deliberate redesign at the level of the program was necessary to effect the systemic changes required to answer this need. We were also mindful to retain our alignment of course and unit-learning outcomes and assessment against the Australian Qualifications Framework specifications.23
The development of a research-based curriculum is not without controversy in the chiropractic community, with detractors maintaining the need for the centrality of the original DD Palmer philosophy,24–26 or who argue that evidence-based chiropractic is severely limited.27 However, the Macquarie University Chiropractic Department is a signatory to the “International Chiropractic Education Collaboration: Clinical and Professional Chiropractic Education Position Statement.”28 This statement explicitly supports an evidence-based approach and places the vertebral subluxation within a historical context. The developments we have made increase the systemic inclusion of these principles and are as such a necessary and welcome step forward.
The following sections elaborate our work on the program and outline our approach and its effectiveness. A review of the literature did not yield any research published on the process of developing a program that focuses on the alignment of course work that enhances evidence-based education. To the best of our knowledge, this paper offers a unique contribution, and our purpose is to share our experience with other medical or allied health programs contemplating a review of curriculum and program redesign.
METHODS
Background to the Process
In 2015–2016 the 5-year chiropractic program (3 years for the bachelor of chiropractic degree, 2 years for the master of chiropractic degree) was redesigned to increase the evidence-based focus of health education from the 1st to the final year of study, at the instigation of the head of department (3rd author). This was done by developing 3 complete streams within the program: a clinical research stream, clinical science stream, and professional practice stream. Each stream forms an essential part of the development of EBP, as it aligns with the 3 integrated aspects of Sackett's6 conception: research (clinical research stream), in combination with the experience (professional practice stream) and expertise (clinical science stream) of the clinician. The intent is to graduate chiropractors that not only achieve accreditation competencies (set by the CCEA) but are evidence based in their approach.
The chiropractic program restructuring fortuitously coincided with a university-wide implementation of a new strategic framework that required program-based design of all programs of study.29 This meant that each program (major or degree) must be designed as a whole rather than an amalgamation of individual courses. As part of program design, the trend at this and other Australian universities has been to move toward program learning outcomes (PLOs) that fit with university-wide graduate attributes.30 While PLOs are still required to respond to specific themes, such as sustainability or employability, each program of study has discretion on how to translate outcomes directly relevant to its graduates. The PLOs for our program were organized into the 3 streams outlined above. In addition, the university framework requires “research and enquiry-led discipline specific program content.” Research-appropriate assessment tasks should be included throughout the curriculum, and research opportunities for undergraduate students should be created, including involvement in research projects.29 The changes in the chiropractic department therefore aligned well at the institutional level, ensuring university support for the changes.
In reviewing the chiropractic program, program-based design, using curriculum mapping and constructive alignment, was employed in a collaborative process called “Design, Develop, Implement” (DDI).31 As we facilitated the process, we used “reflection-in-action”32 to monitor progress and modify activities as needed. This was applied to each of the 3 streams.
The term curriculum mapping was first coined by Heidi Hayes Jacobs33 and refers to the process of setting out the big picture in a way that ensures the curriculum is appropriately taught and assessed. It requires the identification of “exactly what capabilities and competencies are to be developed and assessed in the program overall before ensuring these are mapped to units of study, are validly assessed, validly and reliably graded and the learning methods and resources in each unit of study enable students to optimize their performance on their assessment tasks.”34
Davies et al35 describes this process as a “whole-of-program approach” that focuses on coherent curriculum in which course and program goals or learning objectives are aligned and that uses “constructive alignment (of course-level learning objectives, learning content and assessments).” Once the goals are defined, the relevance of the program can be assessed, and overlaps and gaps are avoided.
Biggs'36 model of constructive alignment is a student-centered approach that starts with learning outcomes and then designs teaching and learning activities and assessments to achieve those outcomes.37 What students do to learn concepts and competencies is important: the assessments must be aligned to the goals, because students learn by doing what they know will be assessed. Knowing this, an authentic assessment that asks students to “perform” their understanding, such as in a clinical situation, will better align with required outcomes for professional practice.38
Work on the program was modeled on the DDI approach for program-level learning design developed at Macquarie University by Vlachopoulos and Seeto31 and practiced with them by the 2nd author. The initial development team used action research as they worked, which means that they used reflection on practice to evaluate and modify the framework for DDI to iteratively improve the process.
Design, Develop, Implement is a team-based approach to university curriculum development that constitutes a flexible set of workshops and principles for program design through hands-on small-group work, facilitated by learning and teaching specialists using “reflection-in-action.” Reflection-in-action acknowledges that problem solving is specific to a situation as it presents and is a “kind of experimenting,” one that is not “controlled” but, rather, “on-the-spot”32 to be processed and reacted to in the moment. Activities are planned based on the framework ahead of time, but as reflective practitioners or “agents/experients” listening to the “back-talk”32 of those activities within a workshop, the team modifies them in light of observed activity. Afterwards, the team reflects on achievements, missteps and gaps and plans the next steps in the process.
Key to DDI's success is a common workspace that encourages all participants to actively add ideas. There are 4 possible implementation models in DDI.31 The model used by the chiropractic department is called DDI Programs, which supports academic teams in developing a new program from existing courses, with an emphasis on developing the philosophy of the new program and aligning and redesigning the existing courses to meet the new program outcomes.31
Program Design Process
The tools for workshops generally include large paper posters adhered to the wall, which are drawn on with marker pens and hold adhesive notes (Fig. 1). Depending on the aims of the workshop, the notes can be color-coded to different functions to give visible clues about distribution and frequency of items. Items are also, importantly, moveable: the aim is to work rapidly and facilitate rearrangement and modification. Workshops also pause at major points in the workshop to share ideas/progress between groups and invite feedback. Similar approaches are employed in other industries and contexts for change and design. The process used here, however, is tailored to the requirements of changing a clinical education program and could be used as a model for other program teams contemplating similar transformation.
Mapping assessment to program learning outcomes (PLOs) for the undergraduate degree, 2nd and 3rd years. The colored strips indicate the PLOs covered by each assessment.
Mapping assessment to program learning outcomes (PLOs) for the undergraduate degree, 2nd and 3rd years. The colored strips indicate the PLOs covered by each assessment.
The department's director of learning and teaching (1st author) worked closely with the faculty facilitation team (which included the 2nd author) to identify priorities and plan relevant workshops. The process began with mapping the existing curriculum. The design process was inclusive of (almost) all members of the department, covering 5 separate half- or full-day workshops and regular smaller meetings throughout 2016. The process of curriculum mapping was made more inclusive by engaging a clinic supervisor, a casual teaching staff member, and a recent graduate of the program, in addition to the permanent academic staff.
All existing courses and their assessments were mapped to the 3 streams in the first 2 workshops. Undergraduate courses could be mapped to the postgraduate PLOs, as they provide the essential scaffolding and building blocks. For example, the courses in human anatomy and the physics and biomechanics of movement have great relevance for chiropractic skills and for neuromusculoskeletal diagnosis.
Once the existing program had been mapped, the next move was to workshop how the learning outcomes should develop within each year and within each stream, changing course learning outcomes and associated student assessments, to better reflect the PLOs. This thus modified the program towards a coherent development of the abilities of the students within the 3 streams.
Clinical Research Stream
In the full-day workshop in July 2016 that specifically addressed the clinical research stream, the team was divided into 3 groups of up to 6 members each, 1 on detailing the research PLOs for each year and semester, a 2nd on mapping assessments identified as related to research against the PLOs, and a 3rd on the culminating research-related courses in the master's degree. Figure 1 shows part of the 2nd group's work on mapping assessments against PLOs (Fig. 1).
For the assessment-mapping group, the wall posters for each year were hung side by side in chronological order, with relevant assessments blue-tacked to them (Fig. 1). Final program learning outcomes from the clinical research stream were color coded, and corresponding colored adhesive notes were used to denote where those PLOs were covered by the assessment. In the process, more assessments were noted as relevant and added with marker pen. It was invaluable to have academics in the room who taught the courses in identifying these details, rather than relying on course guides alone. The size of the strips of color coding for PLOs was a participant modification: smaller strips indicated only a minor contribution to the final PLO. The absence of the color of a PLO on statistical understanding was immediately visible.
Hanging the posters in sequence aided the perspective of seeing the program as a sequenced pathway. This assisted, for example, in recognizing a disjoint in the scaffolding of assessment skills, resulting in moving a premature 2nd-year assessment to a 3rd-year course, with a more straightforward 3rd-year assessment moving into the 2nd year. This was one of the several points when the presence of recent graduates of the program provided student experience-based confirmation and feedback.
The other groups in this day-long workshop, 1 working on detailed PLOs and the other on research-oriented courses in the final year, wrote their notes on large pieces of poster paper, the better to provide a point of collaboration for the group and to share results. The PLO group (see Fig. 2 for an example of their work), faced with writing 10 semesters' worth of learning outcomes, did not have time to complete all PLOs within the allocated workshop time. The team reviewing the research courses were happy with their progress in producing an outline for the revised courses and joined the other 2 groups in the 2nd part of the day.
Program learning outcomes for semesters in the program's 1st year from the clinical research workshop. The codes in brackets indicate which course is expected to cover the learning outcome.
Program learning outcomes for semesters in the program's 1st year from the clinical research workshop. The codes in brackets indicate which course is expected to cover the learning outcome.
After this and other workshops, the facilitation team and the department learning director transferred the results of each workshop into digital spreadsheets and notes, which, together with images of the posters and other aids, were shared with the team. Proposed changes that came from the workshop were highlighted within the notes.
The facilitation team and learning director met regularly to review progress and plan the next steps in the project. To complete and confirm the clinical research PLOs, a further 3-hour workshop was held in September. This workshop was optimistically planned to also start on the other streams, but it became clear in practice that the team needed to spend the whole workshop confirming the important clinical research stream.
Completing the Foundation
A further workshop in December mapped learning outcomes across the program to PLOs in the other 2 streams, clinical science and professional practice. The exercise, as observed in previous workshops, was far from mechanical: the participants were the people who taught the courses and needed to implement any changes. They directly related the learning outcomes to teaching content and assessment in both small-group and full-team discussions. This importantly included discussion on the balance between elements within the overall curriculum, for example, the development of the biopsychosocial model of disease.
RESULTS
Through workshopping the existing program to fit the PLOs better, we developed learning outcomes and associated assessments that progress the PLOs from year to year. Existing assessments for each of the courses were the starting point and were matched to the PLOs on the large posters via color coding (Fig. 1). If they did not fit or fit better in another year, they were moved. Gaps in the assessment of learning outcomes were addressed, and assessments were written to fill the gaps. A process of discussion and consensus led to the charting of assessments for each stream in each year of the program, with the aim of assessing the attainment of the year's PLOs. An example of this can be seen in Table 1, which gives an example of the alignment of learning outcomes to the assessments for the clinical research stream for the master of chiropractic program (Table 1).
Alignment of Learning Outcomes to the Assessments for the Clinical Research Stream for the Master of Chiropractic Program

Changes included minor adjustments as well as major and ongoing adjustments to the curriculum. The undergraduate program was redesigned, resulting in a new schedule of courses, the introduction of a Participation and Community Engagement course (the university's work-integrated learning initiative), the inclusion of a course from the Department of Psychology, and the movement of courses within the program (eg, neuroanatomy and research methods were moved to different years within the program to create a more logical learning pathway for students). The consolidation of subject topics into clinic internship units provided opportunity to allow stand-alone, research-only units to be offered. This greatly assisted assessment of this material in a more authentic manner for both courses.
The learning outcomes for each year and within each stream were finalized, providing a strong foundation for course learning outcomes and assessment design. A timetable of the step-by-step process can be seen in Table 2.
DISCUSSION
The process was often not easy, as workshop dates had to be organized to suit multiple busy schedules. There were often long intervals between workshops, so photographs of shared work taken in the workshops were provided as spatial reminders. The group work on large sheets of paper facilitated the free flow of ideas in the hands-on sessions, but transcription into more formal documents afterwards took effort. The physicality of the shared wall spaces was observed to draw in all participants and hand agency to them, so this format should continue.
The existing 5-year curriculum was useful as a starting point for a map of learning outcomes. The mechanism of curriculum mapping was an effective base for stimulating discussion about the program, opening the teaching that was siloed within existing courses and providing opportunities to address fundamental questions of curriculum and teaching philosophy. The success of the process is vitally dependent on the cooperative presence of the entire department, to provide the details of the courses they teach but also to discuss the flow of information within streams from year to year. It is the team effort that is essential to improving the entire program; as they share practice and details of teaching, each gains a better understanding of the whole. It relies on the goodwill of participants and the flexibility of facilitators in reading the workshops and adapting the agenda “on the fly.”
Planning required participants to balance the value of the learning outcomes within each stream and from year to year. Not all PLOs need the same weighting. For example, the process of developing the skills of differential diagnosis and management begins with the basics of anatomy, physiology, pathology, biomechanics, and techniques. In the later years of the program, this knowledge must be integrated and applied in a clinical context. Other aspects of clinical work, such as providing nutritional advice or understanding the contribution of the mental health of the patient to his or her disease, are also important and require proportional attention in fundamental knowledge and clinical application. This requires care in deciding where and how often these PLOs are taught in the curriculum, to give them due regard but not to overwhelm the student.
Due to staff sabbaticals and leave, it was not possible to include absolutely everyone in the process at all stages, and in some cases at all. It cannot be overemphasized how this process requires cooperation from design to development to implementation. Within the wider framework of accreditation, course instructors at university are able, without consequence, to alter learning outcomes and content; there is little systemic regulation to ensure that courses maintain the original design and place within the program of study. Therefore, it is vital that when an entire program is designed to meet defined learning outcomes for EBP, everyone teaching in the program must understand what the goals are and why they are important. If staff members are left out, or do not participate, or choose not to follow the design, the implementation of the design will fall short of intended outcomes, and at worst it can fall apart—to the detriment of student learning. The problem is persistent, as drift can occur over the years through increments. To be effective, the program requires ongoing monitoring and regular collaboration to continue improvements.
Metaevaluation of the program design should be done to determine the worth of the changes made.39 However, this process is not an easy one. Stufflebeam40 identified 22 approaches to assessing the merit or worth of a program, choosing as most effective those approaches that focus on “not the methods to be employed but either the assessment of value or the social mission to be served.” It seems there is little empirical evidence about the efficacy of the many approaches available.39
Our evaluation will fall with Shufflebeam's “decision/accountability-oriented studies,” which involve a wide range of stakeholders and aim to support improvement, as well as evaluate previous measures. Ongoing metaevaluation of the changes made in the 5-year chiropractic program is best achieved using both quantitative and qualitative methods and formative and summative evaluations. This will include feedback questionnaires and group discussions with staff on the implementation and whether objectives have been achieved. Questions will include “To what extent do program activities follow the program plan?” “To what extent is the program achieving its goal?” and “What problems have you encountered in implementation?”40 Feedback questionnaires could also be directed to final-year students to gauge their experience of the program.
In addition, the effect of these changes will be monitored through regular course evaluations and graduate and employer surveys to see whether they benefit student knowledge and performance and lead to evidence-based practice. The program is also regularly assessed through the accreditation process, which will benchmark it against other chiropractic programs. This should ensure chiropractic remains relevant in the changing landscape of education for health professions, securing its place as an evidence-based discipline.
CONCLUSION
The work outlined here is the big picture of an evidence-based program. Further work is needed to design it into every part of the teaching process. Our intention in the future is to collaborate in smaller subgroups on the learning and teaching methods appropriate to topics within the streams. This includes ensuring that learning methods are appropriate and allow students to adequately perform the assessment tasks.34 Teaching and assessments need to consciously and consistently reflect clinically relevant research that is patient centered. This guarantees that clinical examination is taught within the context of diagnostic test analysis, and clinical management is taught within the context of the latest research into the efficacy of therapeutic, rehabilitative, and preventative regimens.6 The success of the program requires the ongoing cooperation and collaboration of all staff members.
FUNDING AND CONFLICTS OF INTEREST
No funding was required for this research. The authors report no conflict of interest. The authors are solely responsible for the content and writing of this original article.
REFERENCES
Author notes
Stephney Whillier is a lecturer, director of teaching, and postgraduate advisor in the chiropractic department, faculty of science and engineering, at Macquarie University (Balaclava Road, North Ryde, NSW 2109, Australia; [email protected]). Natalie Spence is a senior learning designer, faculty of science and engineering, at Macquarie University (Balaclava Road, North Ryde, NSW 2109, Australia; [email protected]). Rosemary Giuriato is the head of the chiropractic department, faculty of science and engineering, at Macquarie University (Balaclava Road, North Ryde, NSW 2109, Australia; [email protected]). Address correspondence to Stephney Whillier, Chiropractic Department, Faculty of Science and Engineering, Macquarie University, Balaclava Road, North Ryde, NSW, 2109; [email protected]. This article was received December 4, 2017, revised March 1, 2018, and accepted March 29, 2018.
Concept development: RG, SW, NS. Design: RG, SW. Supervision: SW. Data collection/processing: SW. Analysis/interpretation: SW. Literature search: SW, NS. Writing: SW, NS. Critical review: RG, SW, NS.