The objectives for this project were to: (1) identify and agree upon “critical standards” that must be “fully” or “substantially compliant” for a maximum 8-year reaccreditation, (2) compare recent compliance for each critical standard for all accredited programs, and (3) identify which standards have lower compliance levels compared to others.
This qualitative study uses thematic analysis. The 37 European Council on Chiropractic Education (ECCE) standards were assessed by the Quality Assurance Committee (QAC) to identify “critical standards” requiring “fully” or “substantially compliant” ratings for the maximum 8-year accreditation time period. These were approved by the entire ECCE. A table identifying specific criteria for each compliance level then was created. The most recent evaluation reports for all accredited programs were reviewed to record compliance levels for each critical standard and identify the number falling below the status of “substantially compliant”. Specific standards with the highest proportion falling below “substantially compliant” also were identified.
Eighteen of 37 standards were deemed critical. Two of 10 accredited programs had 0 “critical standards” below “substantially compliant” and two programs had three below this level. The most common standard to fall below “substantially compliant” was “faculty recruitment” with three programs (30%) at “partially compliant.”
Identification and approval of “critical standards” requiring at least substantial compliance and the compliance criteria table facilitate implementation of the flexible 8-year reaccreditation period, providing the flexibility needed to work collaboratively with national accrediting agencies. “Faculty recruitment” standard had the highest percentage of programs rated as “partially” compliant.
INTRODUCTION
The European Council on Chiropractic Education (ECCE) is an autonomous organization concerned with accreditation and reaccreditation of institutions offering chiropractic education and training in Europe and South Africa. Accreditation and reaccreditation of institutions is determined by the quality of the chiropractic education and training programs judged against a set of 37 educational standards.1 Currently, 10 chiropractic programs in six different countries are accredited by the ECCE.
As health care education is evolving and improving continuously through evidence-based research,2–8 the standards upon which the educational programs are evaluated also must evolve to reflect these changes.2,8 Regularly updating the educational standards helps to facilitate implementation of current healthcare educational and assessment procedures and processes into the educational programs.8 The frequency of reaccreditation evaluations by the ECCE always has been at 3 years after the first accreditation date and subsequently every 5 years if there were no “concerns” about the program.
Based on a proposal from several stakeholders of the ECCE to allow for a more flexible reevaluation time frame with a maximum accreditation time of 8 years rather than the rigid 5-year reevaluation cycle previously used, the ECCE undertook a thorough review of its 37 standards. The purpose was to determine whether all standards should be weighted equally in determining accreditation status or if specific standards were more important than others and, thus, should be weighted accordingly in deciding the length of the accreditation or reaccreditation time period. Facilitating flexibility in the reevaluation time frames also would facilitate collaborative accreditation/evaluation events with those chiropractic programs also requiring national accreditation in addition to ECCE accreditation. Working together with other accrediting agencies would significantly reduce the time and financial burdens of the accreditation processes on several chiropractic programs accredited by the ECCE.
Therefore, we attempted to: (1) identify and agree on those “critical standards” that must be either “fully compliant” or “substantially compliant” to obtain the full 8-year reaccreditation time frame, (2) document and compare the most recent compliance levels for each of these “critical standards” for each of the 10 accredited programs before officially implementing this procedure, and (3) identify whether certain standards tend to have lower or higher levels of compliance among accredited institutions compared to other standards.
METHODS
Ethics approval for this qualitative study, performed using a thematic analysis method,2,3,6 was not required. First, all 37 ECCE standards were assessed carefully by the ECCE Quality Assurance Consultant (CQA) and those initially suggested to be “critical” for the maximum accreditation time period were selected. This proposal then was sent to the ECCE Quality Assurance Committee (QAC) for evaluation, discussion and revision. Agreement then was achieved between the CQA and QAC on those standards that should be deemed critical. This proposal subsequently was sent to the entire council for discussion and voting at the annual general assembly.
Once the proposal was passed by the general council, a table (Table 1) was created listing the criteria for determining the compliance level for each standard (fully, substantially, partially, noncompliant). This table was based on a similar table used by the United Kingdom's Quality Assurance Agency (shared with the Anglo-European College of Chiropractic [AECC] University College with permission) but modified by the ECCE CQA to focus on a primary health care profession. The proposed table was sent to the QAC for feedback and approval. All 37 standards (critical and noncritical) were to be assigned a level of compliance using the compliance table.
The use of these “critical standards” and the compliance table then was piloted during a site evaluation for one of the programs evaluated by the ECCE with questionnaire feedback obtained after the evaluation visit. The outcomes of this pilot project did not influence the final accreditation decision for this particular institution.
Subsequently, the CQA reviewed each of the most recent evaluation reports for the 10 accredited programs and recorded the level of compliance for each critical standard. A total of the number of “critical standards” falling below the “substantially compliant” level was recorded for each institution/program. The 10 programs currently accredited by the ECCE in alphabetical order are: The AECC University College (United Kingdom), Barcelona College of Chiropractic (Spain), Durban University of Technology – Department of Chiropractic and Somatology (South Africa), Institut Franco-Européen de Chiropraxie (Paris and Toulouse, France), McTimoney College of Chiropractic (United Kingdom), Real Centro Universitario Escorial Maria-Cristina (Spain), Syddansk Universitet Odense – Institute of Sports and Clinical Biomechanics (Denmark), University of Johannesburg – Faculty of Health Sciences – Department of Chiropractic (South Africa), University of South Wales – Welsh Institute of Chiropractic (United Kingdom), University of Zürich – Faculty of Medicine – Department of Chiropractic (Switzerland).
Additionally, each critical standard also was evaluated for the total number of institutions/programs falling below the “substantially compliant” level as well as those “critical standards” with consistently high rates of compliance. Descriptive statistics were applied. Finally, those specific standards having the highest percentage of programs falling below the “substantially compliant” level were evaluated further by the CQA by analyzing the individual evaluation reports from those particular institutions and recording the written justification for the lower compliance level to look for common themes.
RESULTS
Of the 37 accreditation standards, 18 were deemed critical, requiring at least a “substantially compliant” rating to qualify for the maximum 8-year accreditation time period. Appendices A and B list the section numbers and titles of the “critical” and “noncritical” standards, respectively.
The questionnaire feedback from the four members of the site evaluation team piloting the use of these “critical standards” using the newly created criteria table was universally positive. Written comments included that the criteria table gave them more confidence in assigning a compliance level for each standard and that it facilitated higher levels of agreement between the evaluation team members.
For the 10 programs currently evaluated by the ECCE, two (20%) had 0 “critical standards” falling below the level of “substantially compliant.” The highest number of “critical standards” falling below this threshold level was three of 18 (16.7%) and this occurred for two programs (20%). The other six programs evaluated by the ECCE had either one or two “critical standards” below the “substantially compliant” level (Table 2).
The most common standard falling below the “substantially compliant” level was 5.1 entitled (Faculty/Staff Recruitment) with three programs (30%) receiving the “partially compliant” rating. Standards 6.4 (Educational Expertise) and 8.1 (Mechanisms for Program Evaluation) each had two programs (20%) rated as “partially compliant.” Nine of 18 “critical standards” (50%) had no program rated below the “substantially compliant” level (Table 2).
DISCUSSION
Although nearly half of the ECCE accreditation standards were deemed critical, requiring at least the “substantially compliant” rating to achieve the maximum 8-year accreditation time period, this does not mean that the other 19 standards are not important or can be ignored. Many of these noncritical standards are strongly influenced by the setting in which the chiropractic program is imbedded. Stand-alone programs normally would have more control over and, thus, more direct responsibility for some of these noncritical standards compared to programs well integrated within other medical programs (i.e., medicine, other health sciences) that share courses, facilities, processes, and procedures as well as management/governance. Examples would include Standards 6.3 Information Technology, 6.5 Administration and Technical Staff, 4.3 Student Support and Counselling, 1.3 Academic Autonomy, and 2.3 Biomedical Sciences, to name a few. Additionally, the laws governing the chiropractic profession in Europe and South Africa vary widely from country to country and the ECCE standards must be able to accommodate these laws while still protecting the public and profession. For example, chiropractic is still illegal in Spain, whereas it is regulated as one of the five medical professions in Switzerland where chiropractic students also are medical students.
Falling below the “substantially compliant” level for one or more “critical standards” also does not necessarily mean that the program will not be awarded accredited status. A shorter accreditation time period, depending on the over-all outcomes, could be awarded. Based on the outcomes of this study, the ECCE is confident that all 10 programs would still have been accredited, but the actual length of the accreditation for each program may have been different. The number of “critical standards” allowed to fall below the “substantially compliant” threshold that would be allowed for accreditation, but for a shorter time frame, is not currently known as this process is new and only piloted on one institution. Some subjectivity in this process cannot be eliminated, at least initially. The ECCE plans to carefully monitor the outcomes of each accreditation event concerning the use of the table for assigning levels of compliance and length of accreditation awarded, and data will be collected and analyzed concerning the length of accreditation time frame awarded compared to which standards fall and the number of standards falling below the “substantially compliant” level. Data on the consistency of assig ning outcomes between evaluation teams also will be collected. Thus, although identifying these “critical standards” and creating the table listing the various criteria assigned to each compliance level should reduce the subjectivity involved in evaluating chiropractic programs, it will not totally eliminate some subjectivity in this process. Ensuring that all evaluation team members and members of the Commission on Accreditation committee (CoA)1 are well-trained before engaging in the evaluation process is critical to reducing bias and subjectivity. To this end, the ECCE schedules regular face-to-face interactive training workshops, which are posted on the ECCE website.
Table 1 does not list specific examples to help the evaluation team members distinguish between “fully” and “substantially” compliant. Examples of comments that arose for the institution involved in the piloting of these “critical standards” using the compliance table and receiving a “substantially compliant” rating included:
- Standard 1.1:
“The attempt at producing bilingual graduates is proving difficult but still commendable.”
- Standard 1.2:
“Patients are consulted as stakeholders through questionnaires in the clinic, but it is unclear what impact this has on the overall aims and objectives.”
- Standard 2.4:
“There is a need to further develop collaborative procedures/agreements.”
- Standard 2.9:
“Some work is needed to ensure that any overlap is removed and clear remits and boundaries are in place.”
These examples and others gained with further experience will be incorporated into the evaluation team training workshops in the future.
Before implementing this new approach to accreditation using these “critical standards” only two of the 10 chiropractic programs currently accredited had 0 standards falling below the “substantially compliant” level and, thus, would have been awarded the full 8-year accreditation time frame, barring any significant changes to the program occurring during those 8 years requiring an earlier reevaluation. Both programs have been in existence and accredited by ECCE for over 20 years. Both programs having three “critical standards” falling below the “substantially compliant” level were much newer programs and, thus, still on a steeper “learning curve,” with less experienced faculty and staff. Identifying these “critical standards” in advance of their next accreditation event, along with the criteria table, should help these programs better identify those areas requiring improvement and, thus, help facilitate a longer period before reevaluation.
Currently, half of these “critical standards” (9/18) had no chiropractic program falling below the “substantially compliant” level as shown in Table 2. Standard 5.1 (Faculty/Staff Recruitment) was the most common standard to fall below this level with three programs (30%) receiving the “partially compliant” rating. Subsequent analysis of each of the three evaluation team reports for these programs found that the major criticism in all three cases was an inadequate number of full-time faculty. This led to “disproportionate burdens” on existing faculty, which resulted in low research output for the faculty members and, thus, not enough linkage between research and teaching. However, one important advantage of having part-time faculty who also are engaged in chiropractic practice is the level of clinical experience that can be brought to the classroom and incorporated into the student teaching and experience. This is very critical. It also was recognized that for one program receiving the “partially compliant” rating for this standard, the high status of chiropractic within this particular country inhibited chiropractors from pursuing full-time academia due to the large salary difference between practice and teaching. Substantially increasing faculty salaries in this country is not possible due to the fact it is part of the university's faculty of medicine and salary levels are standardized depending on credentials and experience. However, this particular program also was one with a very high level of research output, demonstrating that it is possible to produce relevant research without a high proportion of full-time faculty if the experience and credentials of the faculty are matched appropriately. Perhaps facilitating the incorporation of research into the relevant courses should be part of faculty development rather than requiring all faculty to produce research. Thus, perhaps the ECCE needs to revisit this particular standard and modify the criteria.
To assist accredited programs in their efforts to improve, particularly the area of faculty development and research, the ECCE has created a new position of Quality Assurance Consultant (CQA). One role of this job is to assist new and existing chiropractic programs in their development. Currently, training on the writing and publishing of research studies is occurring for the faculty of one of the chiropractic programs struggling with this issue.
CONCLUSION
The identification and approval of “critical standards” requiring at least the “substantially compliant” level by the ECCE, along with creation and approval of the compliance criteria table now facilitates implementation of the flexible reevaluation/accreditation time period with the maximum of 8 years. This also provides the ECCE with the flexibility needed to work in collaboration with national accrediting agencies to reduce the accreditation workload on chiropractic programs in terms of time and costs. The critical standard with the highest percentage of programs rated as only “partially compliant” was 5.1 (Faculty/Staff Recruitment), with three of the 10 programs receiving this outcome. This is primarily due to inadequate numbers of full-time faculty and their resultant high workload.
FUNDING AND CONFLICTS OF INTEREST
No funding was received for this work and the authors have no conflicts of interest to declare relevant to this work.
REFERENCES
APPENDIX A: CRITICAL STANDARDS
Standards that must be at least “Substantially compliant”
- 1.4
Educational Outcomes
- 2.2
The Scientific Method
- 2.4
Behavioral and Social Sciences, Ethics and Jurisprudence
- 2.5
Clinical Sciences and Skills
- 2.7
Clinical Training
- 2.8
Curriculum Structure, Composition, and Duration
- 3.1
Assessment Methods
- 3.2
Relation Between Assessment and Learning
- 4.1
Admission Policies and Selection
- 4.4
Student Representation
- 5.1
Faculty (Staff) Recruitment
- 6.1
Physical Facilities
- 6.2
Clinical Training Resources
- 6.4
Educational Expertise
- 7
The Relationship Between Clinical or Basic Sciences Research
- 8.1
Mechanisms for Program Evaluation
- 9.2
Academic Leadership
- 9.3
Educational Budget and Resource Allocation
APPENDIX B: NONCRITICAL STANDARDS
- 1.1
Aims and Objectives
- 1.2
Participation in Formulation of Aims and Objectives
- 1.3
Academic Autonomy
- 2.1
Curriculum Model and Educational Methods
- 2.3
Biomedical Sciences
- 2.6
Chiropractic
- 2.9
Program Management
- 2.10
Linkages with subsequent stages of education and training, chiropractic practice, and the Health Care System
- 4.2
Student Intake
- 4.3
Student Support and Counselling
- 5.2
Faculty Promotion and Development
- 6.3
Information Technology
- 6.5
Administration and Technical Staff
- 8.2
Faculty and Student Feedback
- 8.3
Student Cohort Performance
- 8.4
Involvement of Stakeholders
- 9.1
Governance
- 9.4
Interaction with Professional Sector
