Dysphagia is common in individuals with developmental disabilities. Little research exists on the impact of trainings aimed at improving Direct Care Staff's (DCS) use of safe eating and drinking practices. This article presents two studies using pre-and postexperimental design, evaluating online training to improve DCSs' knowledge and ability to identify nonadherence to diet orders. A pilot study (n = 18) informed improvements to the intervention. The follow-up study (n = 64) compared those receiving training with those receiving training plus supervisor feedback. There was no significant difference between groups after training. Both groups increased in knowledge and identification of nonadherence to diet orders. Online training may be an effective tool for training DCS in safe eating and drinking practices.
Hemsley and colleagues (2015) published an article calling on researchers and institutions to investigate dysphagia-related safety among individuals with developmental disabilities (DD). This article went so far as to suggest that research into better ways to prevent premature death in this population, including choking, is an international priority. Dysphagia is an eating and drinking disorder that may affect an individual's ability to position food the mouth and with oral movements such as sucking, chewing, and swallowing (Chadwick & Jolliffe, 2009). Dysphagia is estimated to affect 8% to 50% of individuals with life-long disabilities (Ball et al., 2012; Chadwick & Jolliffe, 2009; Hermans & Evenhuis, 2014; Leslie et al., 2009; Robertson et al., 2017; Sheppard et al., 2014). Those with dysphagia have an increased risk of choking and respiratory infection that can lead to a series of health problems or death if correct protocols around eating and drinking are not followed (Chadwick et al., 2002).
Assisting individuals with dysphagia during meal-time takes specialized training, and maladaptive eating strategies increase risk of asphyxiation and choking (Samuels & Chadwick, 2006). Direct Care Staff (DCS) are usually the primary individuals responsible for implementing safe eating and drinking protocols with individuals with intellectual and developmental disabilities (IDD) in care settings in the United States. They are required to implement meal-time protocols including making modifications to food and drink based on diet orders. Diet orders are instructions on a person's diet or meal time routine and are created by professionals that may include physician, nurse, and/or speech and language pathologist. The diet orders outline (a) what a person can and cannot eat, (b) if a person needs to be in a certain position while eating, and (c) what foods and liquids need to be modified and how they should be modified. Training for DCS on identifying risks in prepared meals as well as body positioning and use of special equipment is needed (Chadwick et al., 2003). Barriers to training DCS include caregiver motivation and lack of time and resources to deliver the training on an ongoing basis (Chadwick, 2017).
Chadwick and colleagues published observational studies of caregiver knowledge and behavioral adherence to written guidelines provided by speech language therapists (Chadwick et al., 2002, 2003). The staff trainings in both studies were comprised of instructional, modeling, and feedback procedures that occurred over an average of 23 months. The researchers assessed knowledge through structured interviews and assessed behavioral adherence though observation. Because the study was primarily observational, the assessments were conducted only after the training, and there were no control conditions or baseline measures. The lack of baseline measures limits the ability to measure improvement over time, and the lack of control condition limits the ability to attribute the assessment results to the intervention. However, in a 2014 study by Chadwick and colleagues, they used both control conditions and baseline measures when evaluating training for caregivers who modified liquids to appropriate safe consistencies for adults with IDD (Chadwick et al., 2014). Sixty-two staff members were randomly allocated to one of the three groups: (a) a control group given written guidance only; (b) a group who received typical training and written guidance; and (c) a third group who received training, written guidance, and the TIM tubes, which are visual aids demonstrating liquid consistency. Participants who used the visual aid had the most improved accuracy in modifying drinks demonstrating that visual aids with typical training were most effective in applying knowledge of modifying drinks.
Recently, a survey was published exploring the current processes, barriers, and solutions to dysphagia management in care settings (Chadwick, 2017). Barriers to adherence were identified as lack of knowledge and understanding of (a) potential risks, (b) food and liquids modification, and (c) the importance of adherence to dysphagia management guidelines set forth by trainers. Barriers also included the lack of time and resources to train and implement effective ongoing training. Stakeholders reported a need for providing information in more accessible ways including using pictures, videos, and models to aid in accurate implementation and to provide feedback on performance. Online interactive training may address some of these barriers.
With the emergence of the Internet, online learning has quickly become a standard for health care education (Irvine et al., 2012; Liu et al., 2016; Ruiz et al., 2006) especially for high-risk interventions where practicing in a virtual environment first, could benefit the safety of the individuals being served (King et al, 2018). However, the impact of online training programs is just starting to be evaluated for staff who work with individuals with developmental disabilities. In 2005, Davis and Copeland evaluated dysphagia related knowledge before and after a computer-based dysphagia training with direct care nurses. The study compared the pre- and postknowledge of an experimental group who received computer-based instruction to a control group who received no training. Results showed that the experimental group demonstrated increased test scores compared to the control group. This study provides preliminary support for the use for online training formats to increase knowledge of dysphagia management, but did not evaluate practice-based knowledge or skills or evaluate combining online training with traditional training methods.
Blended learning is the use of online training in combination with face-to-face training strategies such as coaching and feedback (Bonk & Graham, 2006). Acro (2008) defines feedback as delivering “quantitative or qualitative information used for changing and maintaining specific behavior” (p. 39). Several studies demonstrate the effectiveness of supervisor feedback on improving DCS performance and behavior (Acro, 2008; Ford, 1984; Van Vonderen & de Bresser, 2005), but there is little to no research on supervisor feedback on the implementation of safe eating and drinking protocols or the dosage needed to make supervisory feedback effective.
In this study, we present practice-based research studies investigating the effectiveness of a training package that includes online training modules and then later includes supervisor feedback on DCS's knowledge. We addressed the following research questions:
Can online training increase DCS knowledge of safe eating and drinking practices?
Can an online training increase DCS's ability to identify dangerous situations in meals prepared for the people they serve?
Does supervisor feedback further improve these knowledge and abilities?
The first study was a small pilot study investigating the effect of online training in DCS. The second study was a follow-up study in response to the limitations and findings of the pilot. Improvement to the online training and an additional intervention, supervisor feedback, were added to the follow-up study based on the participant feedback and results of the initial pilot.
Methods: Pilot Study
An online course titled, Bon Appétit: An Overview of Safe Eating and Drinking (Relias, 2016) was developed through Relias, an online health care education company, and was used as the training intervention in the pilot study. The course focuses on teaching safe eating and drinking practices to DCS and can be completed online in 1hour. The course was written by a Doctor of Nursing and reviewed by speech language pathologists and other subject matter experts. The objectives of the course are to (a) implement safe practices to prevent incidents and minimize risk factors during eating and drinking, (b) recognize when a person is choking, (c) interpret and follow diet orders, (d) identify and assess the onset of new problems with swallowing or eating, and (e) follow relevant reporting protocols. The course did not address physical positioning and using specialized equipment. In the course, DCS read through information regarding diet orders, modified diets, food and liquid consistency, high risk foods, and mealtime behaviors through presented scenarios while responding to knowledge checks to ensure engagement. Practice-based lessons allowed learners to identify dangerous situations in the pictures of meals by clicking on parts of the meal that are dangerous. Staff received immediate feedback based on their responses via the course, and reasons why parts of the meal are dangerous are reviewed. They pass the course by completing a 20-question final exam with 80% accuracy. The course can be acquired at ReliasAcademy.com.
The study was submitted and approved by an Institutional Review Board at the Center for Outcomes Analysis. To determine the impact of this course, we conducted a pilot study with 21 DCS in partnership with Easter Seals, a service provider for individuals with disabilities. DCS were recruited from two Easter Seals adult day sites and then randomly assigned to either an experimental group (n = 11) or a control group (n = 10). Participants were mostly female (90%) and had worked as a DCS for an average of 3.2 years. Both groups received a pretest before the intervention phase. After the pretest, the experimental group completed the online training and then immediately completed the posttest. The pre- and posttest were identical except for the order of the questions. The control group completed another online training course unrelated to safe eating (HIPAA Compliance or Abuse and Neglect) and then also completed the posttest. A week later, the control group completed the intervention, Bon Appétit: An Overview of Safe Eating and Drinking. Additional follow-up assessments were completed at 60 and 120 days postintervention. Three participants were removed before analysis because of missing data in either the pretest or posttest.
Assessments were created by researchers and nurses and presented through a computerized multiple-choice assessment (Survey Monkey). The scenario-based questions presented vignettes of individuals with specific diet orders along with high definition pictures of meals. Participants were asked to identify parts of the meals that were not adherent to diet orders and were considered dangerous. Assessments and interventions were conducted online at the place of the employment center during designated training times. The pre-post assessment measured three types of knowledge: (1) Scenario-based diet order adherence questions, 60%; (2) General knowledge 1: Foods—determining which foods are risky, 20%; (3) General knowledge 2: Behaviors—determining which behaviors increase risk (e.g., eating items that are not food, stuffing mouth while eating, etc.), 20%. Responses were combined into one knowledge score based on total percentage correct.
At both 60 days and 120 days after the intervention, participants completed another online assessment presenting different meals and questions regarding risky food and behaviors. These follow-up assessments measured their ability to apply what they learned to the individuals they serve rather than vignettes. The assessments were based on actual individuals' diet orders instead of hypothetical scenarios or general knowledge. The assessment and answer key were created in collaboration with the nurses. Scores were based on percentage correct.
A repeated measures mixed ANOVA was calculated to compare the experimental and control groups across time on the pre- and posttest knowledge assessment. Time (baseline vs. posttest) was the within subject variable and group assignment (experimental vs. control) was the between subjects variable, with knowledge score as the dependent variable. A Cohen's d effect size was calculated using the following equation to examine the training effect (group by time interaction): |(difference between the groups at post) – (difference between the groups at baseline)|/SD of the control group at baseline.
Results: Pilot Study
The repeated measures mixed ANOVA revealed a significant main effect of time, F(1,16) = 20.40, p < .001. This indicates that both groups improved over time. There was not a significant main effect for the intervention group, F(1, 16) = 0.21, p = .65, indicating that both groups performed similarly, and there was no significant group by time interaction, F(1, 16) = 1.33, p = .15. However, for the interaction, there was a Cohen's d effect size of 1.12 indicating that the training enhanced knowledge relative to the controls, and the lack of a statistically significant finding is likely due to low power from the small sample size. A post-hoc effect size calculation was also conducted and with a sample size of 18 participants; we only had .61 power, indicating we would have needed a Cohen's d effect size of 1.40 to see a statistically significant difference. Typically, a study is considered fully powered at .80 or higher. To be fully powered, and achieve a Cohen's d effect size of 1.12, we would have needed a sample size of 28 participants. (See Figure 1.)
To determine if the knowledge gained during training transferred to the DCS's ability to apply this knowledge to individual-specific questions, we assessed the DCS at 60- and 120-day follow up. Both groups demonstrated high levels of individual-specific knowledge at the 60-day (Experimental 82.1%, Control 85.8%) and 120-day (80.2%, 86.2%) follow-up assessments.
Discussion: Pilot Study
Although the assessment results in the pilot study were not statistically significant, the effect size of the interaction comparing the groups over time was quite large with a Cohen's d of 1.12. This result indicates that the group who had the online training (experimental group) performed more than a standard deviation better over time than the control group. The large effect size suggests that there may be a clinically meaningful difference between the groups and that the training did have an impact on knowledge. The 60- and 120-day follow-up assessments were individual specific questions and different from the pre-and postmeasures. We could not compare the pre-and posttests with the follow-up assessments because the follow up assessments measured generalization, but we were able to use the follow-up assessments to improve and inform the assessments for the subsequent study.
Based on the results of this pilot study, we sought to improve the (a) interventions and (b) the assessments. To improve the interventions, we revised the online training by making the course audio-driven, meaning that participants would listen to audio narration of the content rather than reading the course. We did this to ensure that all the information was reviewed and not skipped over. Based on feedback obtained by learners and trainers, we added more opportunities to practice identifying risks within the course and added additional, immediate feedback to responses by explaining why answers were correct or incorrect. Based on further literature review of best practices of DCS training (Bonk & Graham, 2006; Liu et al., 2016; Van Oorsouw et al., 2009) we expanded the intervention in the follow-up study by adding a supervisor feedback component in order to investigate whether blended learning would be more successful in improving participant knowledge and identification of nonadherence to diet orders. We improved the assessments by changing the knowledge assessments to measure only individual-specific knowledge to diet orders rather than measuring general knowledge and scenario-based knowledge. We also added a hands-on interview assessment to measure DCS's ability to identify nonadherence to individual diet orders before and after the training.
Method: Follow-Up Study
We conducted this research in partnership with Easter Seals, a service provider for individuals with disabilities at six adult day sites located in the western United States. The sites were either community-based programs or partial-therapeutic day services for adults with moderate to severe DD. DCS were recruited through posted fliers and supervisor announcements. The pilot study participants were excluded from the follow-up study. DCS who completed the study received $20 Amazon gift card. DCS worked together at each site and were able to freely interact during the training and electronic assessments. Meals occurred on-site or at local restaurants during community outings. Meals for this study were typically prepared by the individual's caregivers at home and sent with them to the day program.
An experimental pre- posttest design was used to compare two groups of DCS. We randomly assigned participants to one of two groups across six locations. One group received the Online Training Only (OTO), whereas the other group received Online Training plus a 20-min Supervisor Feedback (Coaching) session (OT+C). To determine if there was a difference between the groups before and after the interventions, two assessments, one that measured knowledge and the other that measured application of knowledge (detailed in the measures section), were delivered before the intervention (baseline) and then were repeated a week after the intervention (posttest).
Sixty-four participants completed the informed consent and were randomly assigned to one of two groups. We excluded two participants in the Online Training plus Coaching (OT+C) group from the analyses due to missing data in the pretest. Of the 62 participants included in the analyses, 32 were assigned to the courses only group, and 30 were assigned to the courses plus supervisor feedback group. See Table 1 for more participant demographic information.
Electronic Assessment: Identification of Risks
We measured the DCS's ability to identify individual specific risks in meals using an electronic multiple choice assessment. The questions asked DCS to identify foods or items in a picture of a meal (e.g., seeds, straws, garnish, napkins, size of cuts of meat) that are hazardous to the stated individual or client they serve. For example, a high definition picture of a full meal (all items, drink, and utensils) was presented in electronic form to the DCS. The DCS was asked, “What item(s) in this meal are hazardous or need to be modified for (individual's initials)?” They were then required to select which items were hazardous from a list. DCS were familiar with the individuals and their diet orders because they worked with them almost daily and received instructions about their diet orders by a trainer and nurse. Researchers created the assessment in consultation with the nurse who was part of the team that created the individuals diet orders. The nurse created the answer keys which were based on individual's diet orders. Scores reflect the percentage of correct responses to each item listed or how well they could identify which items were hazardous from the pictures. (See Figure 2.)
Hands-On Interview Assessment: Identify Nonadherence
The DCS's ability to identify aspects of meals (size, consistency, type, temperature, etc.) that don't adhere to individual's diet orders was also measured. We asked the DCS to verbally describe what items in a physical meal were hazardous and how they would modify or make the meal safe for a specific individual. This assessment measured not only the participant's ability to identify risks, but also how they would problem solve and alleviate identified risks. This assessment was done in person with two real sample meals in which they could manipulate and observe in detail (e.g., texture, temperature, identify strings or seeds, etc.). There were two versions of each meal (e.g., Meal 1A, 1B, 2A, 2B) that were different but as equivalent as possible. (See Figure 3.)
The investigator interviewed participants one-on-one, presented two meals, and asked the following about each meal,
If this meal is given to (individual), describe what modifications would need to be done to make the meal safe and edible for him/her to eat? Provide specifics on the modifications to size, texture, temperature, and consistency and what needs to be added or removed and any other considerations.
The investigator recorded the participant's responses for each item. An answer key was created by the nurse (who did not administer or score the assessments) from individual diet orders and then the responses were scored by two researchers not involved in the administration of the assessment. The researchers scored the assessments blind to participant or time-point. Items were scored on a 0–2 scale (0 = participant didn't identify anything about the item as a risk and/or provided wrong/unrelated modification, 1 = identified it as a risk but didn't provide complete or most accurate modification, and 2 = identified it as a risk and provided complete and accurate modification). There were two versions of each meal (e.g., Meal 1A, 1B, 2A, 2B) with each version having the same number of meal items with similar types of modifications needed. For example, Meal 1A was peanut butter and jelly sandwich, orange, graham crackers, ice cream cup and water whereas Meal 1B was Nutella sandwich, apple, cookie, jello cup, and water. Participants were randomly assigned version Meal A or Meal B by site for pretest and then assigned the alternate meal for the posttest, for feasibility purposes. Versions were counterbalanced by site so one assessment wasn't used more than another. Final scores reflect the percentage of points the participant answered correctly.
All participants completed the online training, Bon Appetite! An Overview of Safe Eating and Drinking (revised) delivered using Relias' Learning Management System. This location of Easter Seals delivers monthly online training to all staff using this program. The training can be accessed on a computer or mobile device and is audio driven so the computer must have audio features enabled. The objectives for this course are as follows. After taking this course, DCS should be able to: (1) Implement safe practices to prevent incidents and minimize risk factors during eating and drinking; (2) Recognize when a person is choking; (3) Interpret and follow diet orders; and (4) Recognize at a basic level when a person is developing choking risk and report to the appropriate clinician.
In the course, the learner is presented instructions through voice-over, text, and graphics. Every few minutes the learner is asked a competency-based question to keep them engaged. Throughout the course, the learner is presented with scenarios of individual's diet orders then presented with a picture of their meal and asked to identify aspects of the meal that are dangerous. The course provides multiple practice opportunities and immediate feedback upon responses and takes approximately 1 hour to complete. (See Figure 4.)
The supervisor feedback was delivered through one 20-min group feedback session within a week of the online training at the DCS's places of work. Groups were three to six individuals at a time. The purpose of the feedback session was for the supervising nurse to provide feedback to DCS on responses related to identification of risks and ability to adhere to individual's diet orders. In the coaching session, the supervisor presented two sample meals and displayed a poster (see Figure 5) that laid out the important aspects of diet orders they should consider (food's consistency, size, texture, temperature, etc.). She then posed the question: “This meal prepared for (individual they serve) by his/her family. Can this meal be served the way it is or should there be any modifications? If so, what modifications need to be made?” The supervisor provided feedback using the prompt hierarchy in Figure 6.
Results: Follow-Up Study
Baseline Differences Between Groups
T-test analyses and chi-square analyses were conducted to determine if there were any existing differences between the groups at baseline on gender, native language, years of education, years of experience, or baseline ability to identify risks. An independent samples t test indicated no baseline differences between the groups (OTO and OT+C) on baseline scores ability to identify risks, p > .05. A chi-squared analysis also indicated no differences between the groups on native language, years of education, or years of experience, p's > .05). However, there was a significant difference between the groups on gender, with the courses only group containing significantly more men than the courses plus feedback group. Therefore, we included gender as a covariate in the analysis. (See Table 1.)
Electronic Assessment: Ability to Identify Risks
A repeated measures mixed ANCOVA was conducted comparing the groups over time on ability to identify risks, including gender as a covariate. Overall, there was a significant main effect of time, F(1,59) = 22.07, p < .001, but not intervention group, F(1, 59) = 0.60, p = .44, or gender, F(1, 59) = 0.503, p = .86. This finding indicates that there was a difference over time across all groups, and no differences by group or gender. There was not a significant training group by time interaction, F(1, 59) = 0.31, p = .58, indicating that the groups did not differ over time in their performance. Because there was no difference between the groups, we combined them into one group to compare the effect size pre- to posttest, which revealed a medium Cohen's d effect size of 0.58, with the increase in performance pre- to posttest being over half a standard deviation. (See Figure 7.)
Hands-On Interview Assessment: Ability to Modify Meals According to Diet Orders
Forty out of the 64 participants completed this assessment. Twenty-five were not able to participate in this assessment because of staff scheduling and availability. Although not all participants completed this assessment, we believe this measure is extremely valuable because it measured how staff would problem solve and alleviate risks found in meals. Essentially, it measured how the staff would apply their knowledge regarding safe eating and drinking practices.
A repeated measures ANOVA did not reveal a significant group by time interaction on the ability to adhere to individual diet orders, F(1, 39) = 0.08, p = .77, with a small Cohen's d effect size of .07, indicating no difference in how the groups performed over time. However, there was a significant main effect of time across both groups, F(1, 39) = 34.34, p < .001, indicating that both groups significantly improved from pre- to posttest. There was also no main effect of group, F(1, 39) = 2.06, p = .16, indicating that both groups showed similar performance.
Because there was no difference between groups, we combined both groups to compare the effect size pre- to posttest, revealing a Cohen's d effect size of 0.95. This is a large effect size, with the increase in performance pre- to posttest being almost one standard deviation. (See Figure 8.)
Discussion: Follow-Up Study
Electronic and Hands-On Interview Assessments
The electronic and interview assessments found no differences between the OTO and OT+C suggesting that the feedback session did not provide a substantial amount of additional instruction compared to the online training. When the OTO and OT+C groups were combined, the change in average pre-and posttest scores for the interview assessment (15%) was much larger than the change in electronic assessment (7%). This jump in scores decreases the likelihood of a practice effect, as a practice effect would likely affect all tests equally and suggests that the intervention had some impact especially because the time between pre-and posttests was a few weeks apart. Additionally, the pre-and posttest, although equivalent, were not the same, further lowering the likelihood of a practice effect. What the interview assessment measures, that the electronic assessment doesn't, is participants' ability to explain how to modify an existing meal based on an individual's diet orders. This additional measure of staff problem solving and application of knowledge most likely made the assessment more challenging and sensitive for staff.
Participants with years of experience on the job and with basic training in modified diets were only able to identify adherence and nonadherence to diet orders an average of 41% (range 12%–69%) in this assessment. After the intervention, posttest scores averaged 62% (range 13%–94%) This emphasizes the need for ongoing training and supervision and the need for further research to determine what training methods can increase knowledge and DCS behavior regarding diet order adherence. These findings support consideration around adding safe eating and drinking knowledge and skills to national DCS competencies (e.g., National Alliance for Direct Care Staff). Eating and drinking is such an important part of people's lives and DCS play an important role in mealtime practices.
This study provides an example of practice-based research that expands upon the past work investigating effective, practical, and innovative ways to deliver training to support staff on meal modification for individuals with IDD and dysphagia (Chadwick et al., 2002, 2003, 2014, 2017). The assessments, course, and feedback hierarchy could provide an efficient way to train staff on safe eating and drinking practices and provide opportunities to practice before implementing skills with individuals they serve. The assessments and feedback prompt hierarchy could be used as an efficient way to practice skills learned in the course and evaluate how DCS are able to generalize knowledge and skills they learned to diet orders they implement for the individuals they serve.
One limitation of this study is a lack of a control group that did not receive a training intervention, preventing us from ruling out a practice effect. It was not feasible for Easter Seals to have a group that did not receive the intervention. It would be beneficial to rule out practice effects in future studies by staggering the training across time. The authors believe that the combined effect size for both groups (online training and online training plus coaching) shows promise that the online training had a positive effect on both knowledge and ability to adhere to diet orders, but that more research is needed.
Although the effect size was large in pretest to posttest scores for both the electronic and interview assessment, the final scores left much room for improvement. The average posttest score for the interview assessment was 62%, well below what most supervisors would be satisfied with for a skill that directly impacts an individual's health and safety. The responses on the interview assessment were scored on a rating scale instead of a dichotomous scale so the assessment could be more sensitive to different applications of knowledge. Future studies should evaluate ways to increase knowledge and interview scores to a mastery criterion level. We recommend that researchers explore what additional training activities would boost DCS knowledge and skill to a mastery level.
A limitation of the assessments is that we did not assess the actual behavior of modifying a meal. Rather, we assessed the learner's answers about identifying risks and how they would modify a meal based on electronic pictures of meals (electronic assessment) and physical sample meals (interview assessment). There were too many feasibility issues that came with observing actual modification of meals at the locations of care and issues with reliably scoring the implementation of diet orders on the job so the investigators chose to present the assessment in a standard interview format with example meals. Future studies might use fidelity checklists completed by on-site supervisors during observation sessions before and after the intervention.
With the higher risk of choking during meal-times for people with DD, effectively training DCS about safe eating and drinking is imperative to prevent unnecessary emergency events and premature death in this population. This study suggests that an online training program may be effective at improving the ability of DCS to identify and modify meals for the safety of the individuals they serve. As choking and dysphagia continue affecting individuals with DD, evaluation, and development of blended learning training methods to increase accuracy and skills around safe eating and drinking practices may be important components to decreasing the incidence of meal-related emergencies and deaths.
This research was funded by Relias, a health care education company based out of Cary, North Carolina. Thank you to Easter Seals Southern California for collaborating on this research project.