Objective

With the shelter-in-place orders implemented during the COVID-19 pandemic, learning experiences abruptly changed from on campus to wholly online. This qualitative study explores the perceptions and attitudes of students as they adapted their study space, study time, and approach to learning.

Methods

One hundred five students enrolled in a doctor of chiropractic program were invited to participate in a survey to understand how shelter-in-place orders during the COVID-19 pandemic influenced their approach to learning. Fifty-two of 105 (49.5%) students completed the survey. The survey asked students to select their primary study strategy from a list of options and then prompted students to explain how the COVID-19 pandemic influenced their study space, use of technology, study time, and metacognitive cycle of planning, monitoring, and evaluating their approach to learning. A Thematic analysis of the participants' responses was performed.

Results

Nearly all study participants described a challenge in adapting their study space, study time, or approach to learning. Respondents reported that the use of technology did not change because assessments and resources were electronic before the pandemic. Respondents who selected high-impact study strategies such as self-quizzing or who demonstrated evidence of well-developed metacognition described a positive approach to learning more frequently than did respondents who selected low-impact study strategies such as repeated reading or who did not show evidence of metacognitive development.

Conclusion

This study presents student perceptions related to promoting and developing self-regulated learning skills. Educators can use this information to understand the adaptations to changes in learning experiences that may promote successful learning.

The COVID-19 pandemic caused an abrupt change from on-campus to remote learning for students and faculty who had selected an in-person education program. Coursework was presented in a variety of modalities with alteration to content, collaboration, and assessment opportunities. Further, course facilitation occurred asynchronously, synchronously, or in combination. College students expressed stress and cognitive overload in managing the transition to new learning platforms and employing new technology.1  In graduate health sciences professions, students desired face-to-face experiences for clinical skills training and also for the satisfaction of contributing to health care solutions during the pandemic.24  However, because of government orders to shelter in place, campus-based learning experiences such as didactic lecture, student-centered learning activities, and clinical skills training were redesigned as virtual experiences or simulations or were postponed.

An influential learning experience can shape self-set goals and the self-regulated learning strategies used to support these goals.5,6  Positive past experiences that involve feelings of relevance, interest, efficacy, and supportive social structure are more likely to shape self-set goals that drive cognitive, metacognitive, and motivational strategies to pursue mastery of a learning goal.7  When obstacles arise that make it difficult to uphold the learning goal, students who have more positive learning experiences are more likely to employ strategies that support the learning goal.6,8,9  In contrast, students will protect ego goals over learning goals during negative learning experiences. Students are more likely to protect ego goals over learning goals when they are working under externally imposed goals that they may not find valuable nor perceived useful toward a task.

Self-regulated learning can be explained as a 3-layered process in which learning first requires the selection of effective strategies to process information.9  Next, regulation requires the employment of the metacognitive cycle of planning, monitoring, and evaluating to direct the learning process. Finally, students must engage in self-set volitional strategies to protect learning goals from ego goals.911  Students modify their self-set goals based on environmental cues that inform them of expectations and allow them to predict the resource investment needed to achieve their goals. When students experience a challenge in their learning environment, a challenge of learning, or both but are still able to detect forward progress in the mastery of a task, they are more likely to raise their perceived self-efficacy compared with success without challenge.12  When students who are accustomed to face-to-face learning cues are forced to pivot to emergency remote learning, their beliefs about learning and self-efficacy are challenged, and weaknesses in their beliefs are exposed.1 

Online course work requires self-efficacy and well-developed self-regulated learning skills but also requires high-quality course design and service from both the institution and the educator to support a growth mind-set.13  Student attitudes are essential in developing volitional strategies that support self-efficacy and self-regulated learning.14  When the course is designed to facilitate ease of learning and engagement with the material, positive attitudes associated with self-efficacy increase.13  The COVID-19 pandemic induced an emergency remote learning experience, which differs substantially from planned online learning experiences.15  Taken together, emergency remote learning design and attitudes about the emergency learning experience created a source of cognitive overload and stress to manage, among other distractions associated with the COVID-19 pandemic.1 

Students with well-developed self-regulated learning strategies will evaluate environmental cues and successfully adapt to an ever-changing landscape to realize their goals.16,17  Positive learning experiences prompt self-awareness for employing self-regulated learning, but responses to task-specific prompts vary based on the level of cognitive, metacognitive, and volitional development of an individual.18  The demand to develop self-regulated learning skills varies in adulthood due to life experiences.19,20  Educators can effectively support the development of self-regulated learning in a diverse population of students when the level of cognitive, metacognitive, and volitional development is better defined.16 

The purpose of this study was to explore survey responses about how the COVID-19 pandemic shelter-in-place orders influenced students' perception of and attitudes about their approach to learning. It was hypothesized that students who selected high-impact, or active learning, study strategies and used well-developed metacognitive strategies would describe a more positive adaptation to the COVID-19 learning experience compared with students who selected low- impact, or passive learning, study strategies and did not use effective metacognitive strategies. A secondary goal was to identify information about learning strategies, successes, and failures of following through with self-set learning goals so that the data can allow educators to effectively scaffold the development of self-regulated learning skills regardless of the distractions a student faces at a given time.

Participants and Context

Participants were students in their 2nd year of a doctor of chiropractic program at a private not-for-profit university. Demographic data were not included in the survey; however, upon matriculation, the cohort consisted of 45 (36%) students who self-identified as female and 79 (64%) students who self-identified as male, with a mean age of 25 years. One hundred five students registered for a required course were offered the opportunity to participate in an optional and anonymous survey at the end of the academic term occurring from April to June 2020. The survey was administered online through the course learning management system, and data were recorded anonymously. Students were invited to participate in the survey by the course instructor (author J.N.) on June 8, 2020, and again before the closure of the survey. Students completing the survey were offered a 1.5% inducement toward their final course grade. There were no financial incentives to complete the questionnaire. Only students who were 18 years or older, gave informed consent, and completed the survey were included in this study. Fifty-two of 105 (49.5%) students met these criteria. The University of Western States institutional review board declared this study exempt (IRB OIRG 0001188). No funding was provided for this study.

Survey Design

Participants were offered a self-evaluation survey on their perceptions and attitudes regarding their learning experiences and learning environment during the full academic term in which they experienced the shelter-in-place orders (Appendix). The survey had 10 short-answer and 6 open-ended questions asking participants to select and explain how the remote learning experience influenced their study space, technology use, study time, study strategies, influential learning experience, and metacognitive cycle of planning, monitoring, and evaluating their approach to learning. The survey presented a list of 20 study strategies derived from Make It Stick by Peter Brown21  as well as an option to enter another strategy not listed. The choices were presented in random order and not based on high-impact or active learning vs low-impact or passive learning strategies. The study strategy list was used twice: once to ask participants to select all study strategies that they used and next to ask participants to select their primary study strategy. There was no direct pilot of this survey, but it was modified from a separate self-evaluation survey used by the lead authors asking participants to list and explain their primary study strategy, influential learning experience, and metacognitive cycle of planning, monitoring, and evaluating their approach to learning.

Qualitative Data Analysis

Data analysis was performed on the 52 submitted surveys. All surveys were either complete or missing data for no more than 1 item per participant. Any missing data were excluded from the data analysis, and calculations were adjusted where applicable.

The data analysis had a pragmatic approach. Two authors (C.A.W. and J.N.) are involved in educating the students who participated in the survey, and 2 authors (C.B. and B.M.) are educators in separate programs at the institution. Open-ended survey questions were analyzed using reflexive thematic analysis cross-referenced with closed-ended selections. Specifically, the 4 authors used the constant comparative method.22  Open coding involved initial categorization of respondents' explanations for 1 question at a time. Each explanation was compared with closed-ended items to make inferences about their cognitive, metacognitive, and volitional strategies. Next, axial coding occurred in which data for each question were organized in a new way based on connections between categories established in open coding. Finally, in selective coding, the core categories for responses to each question were selected upon systematically relating closed-ended items to open and axial coding.

Survey questions about metacognitive strategies included 3 open-ended questions asking participants to explain how the emergency remote learning experience influenced study planning or goal setting (planning), confidence in study approaches and knowledge (monitoring), and understanding which study approaches were effective (evaluating). Metacognition is a complex thought process that may not be completely captured by a single survey question. Participant responses were scored as sufficient/provides evidence or insufficient/does not provide evidence. Sufficient evidence required information that directly answered the question, while insufficient evidence was a generalized statement or response that was indirect to the question asked. To make inferences about procedural strategies employed by each participant, explanations about study strategies in open-ended questions were compared with closed-ended selections of all study strategies and the primary study strategy. To make inferences about the overall level of metacognitive development, coding using the constant comparative method was applied to participant responses across all questions.

To assess the level of self-perceived volition and adaptation to the emergency remote learning experience, the authors used the constant comparative method with a focus on positive or negative attitudes and actions to the 6 open-ended questions asking study participants to explain the impact on their use of technology, study space, and study time and to explain how the emergency remote learning experience affected the metacognitive cycle of planning, monitoring, and evaluating. At each stage of data analysis, the 4 authors discussed any disagreement and came to a consensus.

Self-evaluation survey responses were examined to explore how emergency remote learning influenced the use of technology, study space, study time, and study habits in 1 cohort of students in their 2nd year of a doctor of chiropractic program. We focused on reporting evidence for the 3 layers of self-regulated learning; cognitive, metacognitive, and volitional strategies. Fifty-two of 105 (49.5%) of surveys met exclusion criteria, as described in the methods.

Primary Study Strategy

Eighty-two percent (42/52) of respondents reported that they did not use the same study strategy as in prior academic terms in their program. Respondents were asked to select all study strategies that they used during shelter-in-place orders from a comprehensive list. The low-impact strategy of repeated reading (36/52) and the high-impact strategy of completing practice problems (34/52) were the most frequently selected strategies when asked to select all strategies used. Other low-impact strategies including highlighting or memorization were also frequently selected (29/52 for each category), whereas other high-impact strategies including self-quizzing (14/52), practice total recall (15/52), or generate explanations (13/52) were less frequently selected.

Next, respondents were asked to select their primary study strategy using the same list. Sixty percent (31/52) of respondents selected low-impact study strategies as their primary study strategy. The most frequently selected strategy, repeated reading, was selected by 23 participants. Eight respondents selected the low-impact strategies rewrite, highlight, flashcards, or memorize. Forty percent (21/52) of respondents selected high-impact study strategies as their primary strategy such as self-quiz, practice problems, generate explanations, create a task list/space study, teach others, or reconstruct the material.

Three respondents included explanations about their study strategies that were not consistent with their self-selected primary study strategy by elaborating on their metacognitive strategies in the open-ended questions. Two respondents selected repeated reading as their primary study approach but explained a high-impact approach by responding to an open-ended question on the survey with the following statement:

Because my study approaches of review and question-answer type of studying have been effective for me, it has shown me that once I've sat down and learned material, I am readily able to apply it.

I typically just reread my material to learn it, and then try to work it into my mental framework by making connections with the new material and others that I have learned.

A 3rd respondent reported their study approach becoming mostly low impact during the pandemic despite selecting self-quiz, a high-impact study strategy as their primary strategy:

Sitting in the same spot, re-reading I was noticing I wasn't grasping much. I was becoming a passive learner and not actively engaging with the material.

Since the survey did not directly ask participants to explain how they use different strategies in their study process, further stratification of study approaches among study participants was not considered. These data imply that although a student selects a high- or low-impact study strategy from a list, it may not reflect the true study approach but rather indicate the 1st step in the approach. Although it is important to disclose information about these outliers from the data analysis, most respondents' explanations aligned with the selected primary study strategy.

Use of Technology

Upon entry to the program, respondents were required to purchase the same electronic device for assessments, and all course material was delivered on a learning management system. The institution also required educators to use the same lecture capture and video conference software both before and during the pandemic. Eight-six percent (45/52) of respondents reported no significant change in the use of technology, and 73% (38/52) described how they used 2 or more electronic devices to view lectures and take notes. Twenty-eight percent (6/21) of respondents who selected high-impact and 19% (6/31) students who selected low-impact primary study strategies specifically described positive adaptations to technology use.

I was also able to cast my lectures to my TV as well so it would feel like I was actually in class, to some degree. (Respondent who selected a high-impact strategy as their primary strategy)

Six respondents specifically described challenges associated with technology, and 4 of these respondents selected low impact as a primary study strategy.

I had to purchase a new keyboard for my iPad because I developed a pretty intense wrist pain from my old keyboard. (Respondent who selected a low-impact strategy as their primary strategy)

Study Space

When shelter-in-place orders were enacted, libraries, coffee shops, and other commonly used study spaces were closed. Seventy-six percent (39/52) of respondents reported that they did not use the same study space during the pandemic compared with prior academic terms. Sixty-one percent (31/51) of respondents indicated a range in level of challenge and adaptability in finding a new study space. One respondent did not make an entry for this open-ended question. The most reported challenges were distractions from housemates and creating a separation of study space from living space. Forty-eight percent (10/21) of respondents who selected high-impact study as a primary strategy and 36% (11/30) of respondents who selected low-impact study as a primary strategy described challenges adjusting to distraction from their study.

Pre-pandemic I studied primarily in the school library. I am now staying at my parent's house studying at the dining table as a makeshift desk. I have less routine and stability while being at home and having more distractions. (Respondent who selected a low-impact strategy as their primary strategy)

I am basically in the same geographic location all day, every day. As a result, it has been incredibly difficult to make the mental transition from work time to study time. In an effort to work around that/fix it, I try and move around the house. (Respondent who selected a low-impact strategy as their primary strategy)

Before the COVID-19 pandemic, my typical study space was at the library, specifically within the group study area. During this last quarter I used my main living space and created an “office area,” moving my desk from my bedroom so I can have some separation between “work” and “home.” I was able to work within a bubble of classmates where we were able to [video] conference, as well as meet up in person within the restrictions of the governor's stay-at-home orders. This was an important piece to be able to continue group studying without feeling like we were putting each other at risk. (Respondent who selected a high-impact strategy as their primary strategy)

Twenty-three percent (12/52) of respondents experienced no change in study space, and some made positive adaptations. Eight respondents who selected low-impact study strategies and 4 respondents who selected high-impact study strategies as their primary strategy described positive adaptations.

I have an office area set aside in my home that I have always used for school. However, I did upgrade my hardware so that I have a larger desktop monitor that connects to my laptop so that I can have better access and use of my school materials. (Respondent who selected a high-impact strategy as their primary strategy)

Study Time

Ninety-four percent (48/52) of respondents reported that they did not use the same amount of time studying during shelter-in-place orders as in prior academic terms in the program. Motivation and efficiency were the most common explanations for why study time changed. Most respondents were not motivated to study because of external factors such as changes in course structure, lack of comradery, or other stressors brought on by the pandemic. Efficiency in study decreased for some respondents. This was directly described in terms of less motivation or efficiency or indirectly by reporting that more study time was needed compared with before the pandemic to overcome lack of focus. Increased screen time and struggles to gain a work-life balance while studying from home were among the commonly described distractions that contributed to less efficient study time.

Trying to study in the midst of pandemic struggles was also overwhelming. Some family members lost jobs, being forced to stay home caused increase tension between some family members . . . are some of the extra stresses that made studying difficult. (Respondent who selected a high-impact strategy as their primary strategy)

The only thing holding me accountable for studying were deadlines or the occasional quiz/exam. I would always study at the last minute because I couldn't get the motivation to study at home during COVID-19. (Respondent who selected a low-impact strategy as their primary strategy)

I feel less motivated being alone and can only bring myself to study as much as necessary to pass. (Respondent who selected a low-impact strategy as their primary strategy)

Now that classes are online, it has significantly increased my screen time. I am also now sitting in the same spot all day. So I find it hard to do school work past 4/5 pm when starting around 7:30 am, but that is barely enough to get through lecture time and readings let alone additional studying. (Respondent who selected a low-impact strategy as their primary strategy)

Other study participants noted increased study efficiency. This was primarily due to the freedom to structure their own learning schedules and ability to create more time for self-care.

I have broken my studying into even smaller sections that I normally would because of my straying focus. So instead of spending an hour on one topic, I spend 15–20 minutes, take a break, and resume for another 15–20 minutes. (Respondent who selected a high-impact strategy as their primary strategy)

This term was magical, [where unlike lecture, the videos] didn't have interruptions from people asking questions. Additionally, I saved time not commuting to school, not spending 10 minutes between each class of campus waiting for the next class the start, etc. I had enough free time this term that I exercised, didn't eat frozen food, slept well each night, rarely studied past dinner. It was simply a gift of time. (Respondent who selected a low-impact strategy as their primary strategy

Thirteen percent (4/30) of respondents who selected low-impact study as their primary strategy and 43% (9/21) of respondents who selected high-impact study as their primary strategy provided evidence of using volitional strategies to make a positive adaptation to use of time. Yet 26% (8/30) and 28% (6/21) of respondents who selected low-impact and high-impact study as their primary strategy, respectively, directly described the lack of motivation as the primary reason that their study time changed (see the sample quotes above).

Given the freedom of online school, I condense my studying into the first 3 days of the week and then I often spend more time doing other things or reviewing on Thursday and Friday. (Respondent who selected a high-impact strategy as their primary strategy)

Forty-eight percent (10/21) of respondents who selected high-impact study strategies as their primary strategy and 70% (21/30) of respondents who selected low-impact study strategies as their primary strategy described negative feelings about the challenges associated with finding time to study, such as feeling constantly behind, frustrated with doing the bare minimum, and being generally anxious.

Metacognitive Strategy: Planning

Survey participants were asked to explain how the emergency remote learning experience influenced the way they plan to study or set new goals. Sixty-one percent (32/52) of respondents provided sufficient evidence for how plans or goals changed. The most common plan, described by 18 respondents, was to structure study with task lists and/or to space study to help improve their approach to learning. All respondents providing sufficient evidence for making new study plans or setting new goals described the need to employ high-impact learning strategies such as case evaluation or self-quiz, whether they selected low-impact or high-impact strategies as their primary strategy.

I've found that, even though I'm usually already diligent about being well organized and planned out for the week, that being remote has made me anxious about keeping up with everything. Planning the studying during the week has been a challenge, mostly because organizing everything that's coming up has been difficult, and a couple quizzes or assignments have caught me by surprise. Coming out of this pandemic, I will be trying my best to have a full weekly checklist done and make sure that I am aware of every project, assignment, or test that is upcoming. This will reduce anxiety and ensure that everything is submitted on time and has my best work put into it. (Respondent who selected a low-impact strategy as their primary strategy)

With the majority of our work becoming asynchronous lecture or asynchronous case/question-based work, I found that I had to force myself to set a more rigorous schedule to stay on top of things. The case-based questions I took very seriously and challenged myself to see how I would react in that situation. I think going forward into future quarters, I plan to create a similar schedule for myself, as well as challenge myself by seeking out case-study questions to try and test my critical thinking in specific scenarios. (Respondent who selected a high-impact strategy as their primary strategy)

Forty-five percent (14/31) of respondents who selected a low-impact study strategy as their primary strategy described a positive or solutions-oriented plan moving forward, while 71% (15/21) of respondents who selected a high-impact study strategy as their primary strategy described a positive or solutions-oriented plan moving forward.

I have never been the most tech-savvy person, so the amount of online material could sometimes be overwhelming. I would have to write out every course on my white board at the start of each week just so I could visualize every course that needed to be looked at, and I would write what I needed to do for each course that week next to it. Having all this stuff on a white board also helped me prioritize what classes needed the most attention any given week so it helped my ability to set goals for the week. (Respondent who selected a high-impact strategy as their primary strategy)

Half of respondents (10/20) who did not provide sufficient evidence for how their study plans or goals changed described challenges associated with emergency remote learning with negative statements or generalized frustration, while the other half had neutral statements describing challenges of remote learning.

The novel challenges that I met during the pandemic lowered my studying expectations. It's sad, but it's reality. (Respondent who selected a low-impact strategy as their primary strategy)

I feel as if the studying and learning quality has significantly decreased and I am more “checking off tasks” as there is so much to accomplish that is self-directed/self-motivated. It is hard to push beyond just getting through tasks and actively take the extra time to learn and study effectively. The remote learning experience is exhausting and mentally draining. (Respondent who selected a low-impact strategy as their primary strategy)

Metacognitive Strategy: Monitoring

Study participants were asked to explain how the emergency remote learning experience influenced the way they measured their confidence in their study approach or knowledge. Seventy-five percent (39/52) of respondents provided sufficient evidence for monitoring their study approach or knowledge. Twenty-three percent (12/52) of respondents reported decreased confidence in their clinical skills due to lack of hands-on practice with professors and peers. Eleven percent (6/52) of respondents had decreased confidence in long-term retention of knowledge due to the shift to online learning and pandemic-related stressors.

My confidence definitely took a hit. Especially with the starting clinic. This is easily one of the most important quarters to gain confidence, and I feel like we missed out on a lot and being able to properly be prepared. (Respondent who selected a high-impact strategy as their primary strategy)

Across the board, my confidence that I am learning as much and as well as I'd like to be dropped precipitously due to the pandemic. My confidence that I could use mostly the same study approaches to achieve the same grades was more or less the same. But I really feel like the decreased amount of effort that I had to put in all quarter will negatively impact my long-term mastery of all of the content of this quarter. (Respondent who selected a low-impact strategy as their primary strategy)

Sixty-one percent (19/31) of respondents who selected low-impact study strategies as their primary strategy reported decreased confidence in their study approach or knowledge, and 22% (7/31) of respondents reported increased confidence. For participants who selected high-impact study strategies as their primary strategy, 43% (9/21) of respondents reported increased confidence and an equal percentage reported decreased confidence. Notably, 43% (9/21) of respondents who selected high-impact study strategies had a solutions-oriented approach to monitoring their study approach and knowledge regardless of their level of confidence, while only 16% (5/31) of respondents who selected low-impact study strategies displayed evidence of a solutions-oriented approach.

The remote learning experience made me realize how reliant I am on structure in school. I am not confident in my ability to teach material to myself or create my own schedule, and in the future, if remote learning were to continue, I would need to make sure I do create a routine that I stick to. (Respondent who selected a low-impact strategy as their primary strategy)

I think when this term started, my confidence was pretty low in terms of thinking that I was going to be learning anything, but that feeling of uncertainty gave me motivation to do everything I could to learn the material and to do well. Due to this regimented and rather intense approach, I did find myself hitting burnout for a longer stretch of time. So, I went from feeling not too confident to very confident but feeling very burnt out. (Respondent who selected a high-impact strategy as their primary strategy)

Uncertainty was another theme that emerged in comparing statements about monitoring between study participants who selected low-impact or high-impact strategies. Thirteen percent (4/31) of respondents who selected low-impact study strategies were uncertain about their study approaches, while 10% (2/21) of respondents who selected high-impact strategies expressed uncertainty.

I feel like I am second-guessing my knowledge more than before this pandemic, as I never know if I am dedicating enough time to each class. (Respondent who selected a low-impact strategy as their primary strategy)

I don't think the level of confidence I have in my study approaches changed all that much due to the pandemic. The way I studied may have changed a bit, but I always make sure I do what I have to be prepared for a class. We will see what happens when we get evaluated on our hands-on skills. I have been practicing and am optimistic I will be up to par, but I have no idea what to expect. Just gotta keep practicing. (Respondent who selected a high-impact strategy as their primary strategy)

Metacognitive Strategy: Evaluating

Study participants were asked to explain how the emergency remote learning experience influenced the way they evaluated their study approaches, and half of them (26/52) provided sufficient evidence for evaluating their studies. Seventy-seven percent (20/26) of respondents described that high-impact strategies were more effective than low-impact strategies, and 23% (6/26) described how stressors caused them to revert to less effective low-impact strategies.

I felt that my ability to study how I would in a typical environment of highlighting and going over the material was all but gone. I did find that question/case-based assignments really helped me figure out the material and critically think. This only worked if I did it closed book. As soon as I started to look up the resources without challenging myself, my learning stopped. (Respondent who selected a high-impact strategy as their primary strategy)

COVID made it a lot harder to connect with other classmates to study, which is usually how I studied before. Everyone has such different schedules and are doing class lectures at different paces. That is normally my main way of learning, so I have had to do a lot more self-quizzing. (Respondent who selected a low-impact strategy as their primary strategy)

Re-reading notes wasn't as helpful. Sitting in the same spot, re-reading I was noticing I wasn't grasping much. I was becoming a passive learner and not actively engaging with the material. (Respondent who selected a high-impact strategy as their primary strategy)

Some statements without sufficient evidence to support evaluation of study approaches described that high-impact strategies and structured study are important for success, but the participant did not indicate whether the strategy was employed. These descriptions were not sufficiently consistent with other selections on the survey to make an inference about the actual strategies used. Other statements reiterated that distractions and stressors decreased the capacity to learn.

Remote learning forces the student to take more responsibility for the education they are receiving and integrate their lectures into applicable knowledge for future practice. (Respondent who selected a low-impact strategy as their primary strategy)

I don't feel I'm in a position to critique what study approaches worked this quarter because I feel that I've been barely studying. I feel that I want to just get through the quarter with the bare minimum and I'm almost okay with that. I've never felt so unmotivated with my studies throughout all my education. I know that if I continue doing the bare minimum in terms of studying, my knowledge of the material will suffer for board exams. and I will not be at the level of intelligence I want to be at when treating my future patients. (Respondent who selected a low-impact strategy as their primary strategy)

Several themes emerged in comparing statements among study participants who selected low-impact and high-impact study strategies. These included the ability to adapt learning; lack of adaptation due to less capacity, discipline, or motivation; and low self-agency. Forty-five percent (14/31) of respondents who selected low-impact study strategies provided evidence of positive adaptations to remote learning, while 67% (14/21) of respondents who selected high-impact study strategies positively adapted to remote learning. Twenty-six percent (8/31) of respondents who selected low-impact study strategies did not have evidence that they adapted their learning, whereas 24% (5/21) of respondents who selected high-impact study strategies did not adapt. Three respondents (10%) who selected low-impact study strategies described low self-agency by indicating that the professor or school should direct their learning, whereas no students who selected high-impact study strategies demonstrated evidence of low self-agency.

Reading and memorizing don't work for me, but they seem to be the only way material is presented to us. (Respondent who selected a low-impact strategy as their primary strategy)

Educators are encouraged to enhance the self-regulated learning and metacognitive skills of their students to promote critical thinking. Well-developed learning skills offer a framework by which to adapt to an ever-changing landscape. The COVID-19 pandemic created a powerful disruption to education that challenged procedural, regulative, and volitional skills. In this study, students in a graduate health sciences program were asked how the COVID-19 pandemic emergency remote learning experience influenced their use of technology, use of study space, and study habits.

The degree of acceptance by the user as well as the effective use of technology make the transition to online learning more successful.23,24  Participants in this study experienced few, if any, issues in technology associated with the transition to online learning because they were accessing course material from an online platform prior to the COVID-19 pandemic. When shelter-in-place orders were enacted, many students left campus and sheltered in place with family or friends. The abrupt move caused substantial distractions with either decreased time alone or too much time alone and challenged the work-life separation.

Evidence from this exploratory student perceptions study suggests that students with well-developed self-regulated learning skills measured by study strategy selection, and evidence of planning, monitoring, and evaluating their approach to learning, were more likely to make positive, solutions-oriented adaptations to the emergency remote learning experience than students with less developed self-regulated learning skills.

Nearly all survey participants reported that their study strategies and time invested in studying changed during emergency remote learning. Using student perceptions data, like those presented in this report, and cross-referencing best practices reported in the literature, educators can design a data-driven positive learning experience. To support a thoughtful learning experience, educators should create opportunities for students to construct and evaluate knowledge instead of reproducing the information provided.25  Educators can enhance procedural knowledge by writing learning objectives for each learning experience that establishes a learning goal and is aligned with course or program goals.26  Designing a progression of naïve to informed tasks will allow the learner to discern what they know and what they have questions about so they can use their time and resources to effectively accomplish their learning tasks.27  Respondents in this report indicated that case evaluations and exams that elicited a higher level of cognitive behavior drove positive adaptations to their learning, while courses with less communication or organization created more obstacles to learning.

To explore student perceptions of how the COVID-19–induced emergency remote learning experience influenced metacognitive skills, study participants were asked how this experience influenced planning and goal setting, confidence in their study approach and knowledge, and evaluation of the study approach. One-third to one-half of respondents did not directly answer the questions posed and instead described specific challenges that distracted them from their learning goals. It is regrettable that study participants were not directly queried about how the COVID-19 pandemic influenced their volitional strategies because it is an essential component to self-regulated learning.79  However, nearly all study participants described an obstacle and explained how they responded to the obstacle, with or without sufficiently answering the question posed in the survey. From these statements, it was possible to make inferences about the ability to protect learning goals over ego goals. Negative experiences challenge follow-through with learning goals and stimulate the protection of ego goals.8  The COVID-19 pandemic induced complex challenges to nearly all aspects of life, making learning goals difficult to pursue even with the most advanced volitional strategies. Several respondents articulated disappointment in straying from their learning goals and described the difficulty they anticipated in reestablishing study strategies consistent with their learning goals. These respondents acknowledged that their study approach was inferior under the shelter-in-place orders, but they either hoped their approach was sufficient to progress to the next academic quarter or they described a plan to make up for lost time and understanding. Similarly, another study of upper division undergraduate students with well-developed metacognition described how they reverted to low-impact study strategies such as repeated reading during times of stress.28  The upper-division undergraduate students explained that they knew they would have to revisit the material using high-impact study approaches such as self-quizzing but that the low-impact studying was better than not studying at all during the stressor.28  While there was evidence of making adaptations to favor a growth mind-set across all categories described in this report, a higher percentage of respondents using both high-impact study strategies and providing evidence of well-developed metacognitive skills also described solutions-oriented adaptations to their stressors. This finding supports the observation that when a student invests in the development of all 3 levels of self-regulated learning, the likelihood of making a positive adaptation to a challenge increases.911 

Although 54% of respondents in this study reported decreased confidence in learning during emergency remote learning, this is not a novel concept. Several studies describe a gap in confidence and the value in building confidence to achieve learning goals.14,29  Educators can design a thoughtful learning space that promotes development of metacognitive and volitional strategies by modeling complex thought.25  Posing questions during or immediately after a learning experience allows the learner to directly reflect on the activity. Using debriefs of case evaluations, assignments, or formal assessments that challenge the learner to explore where their uncertainties lie and how to address them in the future will develop their metacognitive skills of planning, monitoring, and evaluating.30,31  Creating a safe space to learn by both facilitating and joining in an equal conversation will allow students to take risks in demonstrating their thinking and appreciate the value in learning from mistakes.9,25 

In this study, respondents reported higher confidence when they had an opportunity to engage with their instructor and peers or made plans for more time to engage in case evaluations and practice problems to increase their confidence in the future. The COVID-19 pandemic crisis is often described as a transformative event in health care that includes health care education.32  If we view learning through the lens of transformative learning described by Mezirow, we understand that transformative learning begins with a disorienting dilemma that often elicits discomfort.33  Whether these feelings stem from the stressors of being in a health care professional program or are confounded with environmental stressors, decisions are made to manage learning goals and ego goals. As a learner's frame of reference is challenged, a safe learning environment facilitates the acceptance of vulnerability and willingness to engage that are needed to progress through the transformative learning process. Learning spaces that support autonomous thinking and self-empowerment elicit an intentional awareness of both metacognitive and volitional strategies, which support the transformative learning process.34 

While this report provides rich data about how the COVID-19 pandemic emergency remote learning experience influenced students' approach to learning, the most important limitation of this study is the interpretation of student perceptions. Limited verbalization about how the learning experience influenced their study strategy may not reflect the true intent of the respondent. Perhaps because of the sensitivity and stressful nature of the questions' subject matter, many respondents focused on keywords such as study space, study time, goal, or confidence and described how the challenges associated with the shelter-in-place orders were difficult to overcome, instead of explaining specific influences related to their study approach.

Because this study involved a relatively small sample of students taken from a single academic cohort, these findings are limited in their generalizability to other populations or students in other programs or academic disciplines. The nature of the survey as a nonvalidated, qualitative instrument further contributes to this limitation, potentially reducing the reliability of the results and precluding inferences regarding causality or strength of association among the studied variables.

Another limitation may be in selecting a primary study strategy from a prepopulated list. Although the strategies were presented in a random order and we asked study participants to select all study strategies that they used before asking students to select their primary study strategy, the order of options may have influenced their primary strategy selection, and it may not be their primary strategy. For example, 3 respondents selected repeated reading as their primary strategy and yet described in later questions how they primarily generate explanations or integrate new ideas into past or concurrent content for understanding. However, the multiple analyses of the data by all authors were intended to provide the closest possible reflection of the respondent's intent.

In an emergency remote learning situation, whether spurred by a natural disaster or a pandemic, both students and educators are required to adopt new strategies to achieve successful learning. The student must align their learning goals with the activities and approaches used by the educator,10  while the educator must develop viable pathways to learning using different, and sometimes limited, resources. By explicitly focusing on effective self-regulation of learning in the virtual classroom, educators can minimize the barriers to achievement imposed by emergency remote learning situations. This emphasis may also enhance the educator's awareness of their own learning strategies and challenges, enabling them to better support students' development.35 

When coursework includes opportunities to exchange ideas and debrief during and after individual learning experiences, students' metacognitive and volitional strategies are more likely to improve.9,12  When communication is clear regarding learning objectives and how resources may be used, students' procedural effectiveness is enhanced.25,26  Taken together, these elements represent student-centric practices in education that may be applied in all environments, even under challenging and disruptive circumstances.

This study presented a thematic analysis of student attitudes and perceptions regarding changes in resources and study habits during shelter-in-place orders in the COVID-19 pandemic. Students offering evidence of well-developed self-regulated learning skills more frequently described positive adaptations to learning during the pandemic than did students who did not offer evidence of well-developed self-regulated learning skills. Furthermore, most study participants described the value of self-quizzing and engaging in the evaluation of clinical scenarios to enhance their learning skills. These findings suggest that by promoting the use of high-impact learning strategies and facilitating students' metacognitive development, educators may help students adapt to challenging changes in their learning environment. Further investigation is needed to determine whether these findings are generalizable to students in other populations and academic disciplines and to other types of disruptions to the learning environment and to clarify whether a causal relationship exists between metacognitive development and successful adaptation to a disrupted learning environment.

The authors received no funding for this research. The authors have no conflicts of interest to declare relevant to this work.

1. 
Patricia
A.
College students' use and acceptance of emergency online learning due to COVID-19
.
Int J Educ Res
.
2020
;
1
:
100011
.
2. 
Compton
S,
Sarraf-Yazdi
S,
Rustandy
F,
Radha Krishna LK. Medical students' preference for returning to the clinical setting during the COVID-19 pandemic
.
Med Educ
.
2020
;
54
(10)
:
943
950
.
3. 
Kim
SM,
Park
SG,
Jee
YK,
Song
IH.
Perception and attitudes of medical students on clinical clerkship in the era of the coronavirus disease 2019 pandemic
.
Med Educ Online
.
2020
;
25
(1)
:
1809929
.
4. 
Abbasi
S,
Ayoob
T,
Malik
A,
Memon
SI.
Perceptions of students regarding e-learning during COVID-19 at a private medical college. Pak J Med Sci.
2020
;
36(COVID19-S4):S57–S61.
5. 
Fredrickson
BL.
The broaden–and–build theory of positive emotions
.
Philos Trans R Soc Lond B Biol Sci
.
2004
;
359
(1449)
:
1367
1377
.
6. 
Fernandez-Rio
J,
Cecchini
JA,
Méndez-Gimenez
A,
Mendez-Alonso
D,
Prieto
JA.
Self-regulation, cooperative learning, and academic self-efficacy: Interactions to prevent school failure
.
Front Psychol
.
2017
;
8
:
22
.
7. 
Boekaerts
M,
Corno
L.
Self-regulation in the classroom: a perspective on assessment and intervention
.
App Psychol
.
2005
;
54
(2)
:
199
231
.
8. 
Boekaerts
M,
Cascallar
E.
How far have we moved toward the integration of theory and practice in self-regulation?
Educ Psychol Rev;
2006
;
18
(3)
:
199
210
.
9. 
Boekaerts
M.
Cognitive load and self-regulation: attempts to build a bridge
.
Learn Instru
.
2017
;
51
:
90
97
.
10. 
Boekaerts
M.
Self-regulated learning: a new concept embraced by researchers, policy makers, educators, teachers, and students
.
Learn Instr
.
1997
;
7
(2)
:
161
186
.
11. 
Boekaerts
M.
Self-regulated learning: where we are today
.
Int J Educ Res
.
1999
;
31
(6)
:
445
457
.
12. 
Bandura
A.
Self-efficacy: toward a unifying theory of behavioral change
.
Psych Rev
.
1977
;
84
(2)
:
191
215
.
13. 
Heckel
C,
Ringeisen
T.
Pride and anxiety in online learning environments: achievement emotions as mediators between learners' characteristics and learning outcomes
.
J Comput Assist Learn
.
2019
;
35
(5)
:
667
677
.
14. 
Zimmerman
BJ.
Becoming a self-regulated learner: an overview
.
Theor Into Pract
.
2002
;
41
(2)
:
64
70
.
15. 
Hodges
C,
Moore
S,
Lockee
B,
Trust
T,
Bond
A.
The difference between emergency remote teaching and online learning. Published
2020
.
16. 
Williams
CA,
Takaki
LAK,
LeFebvre
R.
Measuring the level of metacognitive regulation in graduate health sciences students: what is the value of a prompt?
Med Sci Educ
.
2019
;
29
(2)
:
409
418
.
17. 
Stanton
JD,
Neider
XN,
Gallegos
IJ,
Clark
NC.
Differences in metacognitive regulation in introductory biology students: when prompts are not enough. CBE Life Sci Educ.
2015
;
14(2):ar15.
18. 
Kuhn
D.
How do people know?
Psychol Sci
.
2001
;
12
(1)
:
1
8
.
19. 
Veenman
MVJ,
Spaans
MA.
Relation between intellectual and metacognitive skills: age and task differences
.
Learn Individ Differ
.
2005
;
15
(2)
:
159
176
.
20. 
Kuhn
D.
Metacognitive development
.
Curr Dir Psychol Sci
.
2000
;
9
(5)
:
178
181
.
21. 
Brown
PC.
Make It Stick: The Science of Successful Learning. Belknap Press;
2014
.
22. 
Creswell
JW,
Poth
CN.
Qualitative Inquiry and Research Design: Choosing Among Five Approaches. 4th ed. Sage;
2018
.
23. 
Bower
M.
Technology-mediated learning theory
.
Br J Educ Technol
.
2019
;
50
(3)
:
1035
1048
.
24. 
Tarhini
A,
Hone
K,
Liu
X,
Tarhini
T.
Examining the moderating effect of individual-level cultural values on users' acceptance of e-learning in developing countries: a structural equation modeling of an extended technology acceptance model
.
Interact Learn Enviro
.
2017
;
25
(3)
:
306
328
.
25. 
Persky
AM,
Medina
MS,
Castleberry
AN.
Developing critical thinking skills in pharmacy students
.
Am J Pharm Educ
.
2019
;
83
(2)
:
7033
.
26. 
Medina
MS,
Castleberry
AN,
Persky
AM.
Strategies for improving learner metacognition in health professional education
.
Am J Pharm Educ
.
2017
;
81
(4)
:
78
.
27. 
Roberson
B,
Franchini
B.
Effective task design for the TBL classroom
.
J Excell Coll Teach
.
2014
;
25
(3–4)
:
275
302
.
28. 
Dye
KM,
Stanton
JD.
Metacognition in upper-division biology students: awareness does not always lead to control. CBE Life Sci Educ.
2017
;
16(2):ar31.
29. 
Dinsmore
DL,
Alexander
PA,
Loughlin
SM.
Focusing the conceptual lens on metacognition, self-regulation, and self-regulated learning
.
Educ Psychol Rev
.
2008
;
20
(4)
:
391
409
.
30. 
Chen
P,
Chavez
O,
Ong
DC,
Gunderson
B.
Strategic resource use for learning: a self-administered intervention that guides self-reflection on effective resource use enhances academic performance
.
Psychol Sci
.
2017
;
28
(6)
:
774
785
.
31. 
Watson
CE,
Bowen
JA.
Teaching Naked Techniques. John Wiley & Sons;
2017
.
32. 
Casale
M.
COVID-19: Can this crisis be transformative for global health?
Glob Public Health
.
2020
;
15
(11)
:
1740
1752
.
33. 
Mezirow
J.
Contemporary paradigms of learning
.
Adult Educ Q
.
2016
;
46
(3)
:
158
172
.
34. 
Van Schalkwyk
SC,
Hafler
J,
Brewer
TF,
et al.
Transformative learning as pedagogy for the health professions: a scoping review
.
Med Educ
.
2019
;
53
(6)
:
547
558
.
35. 
Moos
DC,
Ringdal
A.
Self-regulated learning in the classroom: a literature review on the teacher's role. Educ Res Int. 2012:Article ID 423284.

APPENDIX A

Self-Evaluation of How Remote Learning Experiences During the COVID-19 Pandemic Have Influenced My Study Habits

Please answer the following questions honestly and completely.

This survey is anonymous and is for educational research. The survey has no impact on my evaluation of your performance.

  1. Do you use the same study strategies in the COVID-19 pandemic that you used in prior terms as a doctor of chiropractic (DC) student?

    • Yes

    • No

  2. Do you use the same study space in the COVID-19 pandemic that you used in prior terms as a DC student?

    • Yes: Please explain how you retained your primary study space.

    • No: Please explain how your primary study space has changed.

  3. Do you use the same technology in the COVID-19 pandemic that you used in prior terms as a DC student?

    • Yes: Please explain how technology use has stayed the same.

    • No: Please explain how technology use has changed.

  4. My studying is (check one):

    • distributed across weeks

    • done in a 24–48-hour period

  5. Do you use the same amount of time studying in the COVID-19 pandemic that you used in prior terms as a DC student?

    • Yes: Please explain why the time you study did not change.

    • No: Please explain why the time you study changed.

  6. What kind of strategies do you use to study during the COVID-19 pandemic? (Check all that apply)

    • Reread/rewatch lecture material

    • Complete practice problems

    • Create outlines, concept maps, or flowcharts

    • “Memorize”

    • Mnemonics or memory palace

    • Practice total recall

    • Flashcards

    • Teach others

    • Study with a group of students

    • Rewrite notes

    • Highlight

    • Generate explanations

    • Stop and relate new material to other material

    • Create real-life examples

    • Space your studying

    • Self-quiz

    • Mix the content of study periods

    • Tie to big picture

    • Reconstruct or reorganize the material

    • Structure studying by creating a task list

    • Another strategy

  7. What is the primary strategy you use to study during the COVID-19 pandemic? (Check one)

    • Reread/rewatch lecture material

    • Complete practice problems

    • Create outlines, concept maps, or flowcharts

    • “Memorize”

    • Mnemonics or memory palace

    • Practice total recall

    • Flashcards

    • Teach others

    • Study with a group of students

    • Rewrite notes

    • Highlight

    • Generate explanations

    • Stop and relate new material to other material

    • Create real-life examples

    • Space your studying

    • Self-quiz

    • Mix the content of study periods

    • Tie to big picture

    • Reconstruct or reorganize the material

    • Structure studying by creating a task list

    • Another strategy

  8. Did the remote learning experiences during the COVID-19 pandemic influence you to change your study habits?

    • Yes

    • No

  9. Please explain how the remote learning experiences that you experienced during the COVID-19 pandemic changed the way that you plan to study or set new goals.

  10. Please explain how the remote learning experiences that you experienced during the COVID-19 pandemic influenced the way that you measure the level of confidence that you have in your study approaches and knowledge.

  11. Please explain how the remote learning experiences that you experienced during the COVID-19 pandemic helped you understand what study approaches worked and did not work.

This is an award-winning paper presented at the Chiropractic Educators Research Forum (CERF), Harnessing the Web: How Chiropractic Education Survives and Thrives During the COVID-19 Pandemic. The CERF Awards are funded in part by sponsorships from NCMIC, ChiroHealth USA, Activator Methods, Clinical Compass, World Federation of Chiropractic, and Brighthall. The contents are those of the authors and do not necessarily represent the official views of, nor an endorsement, by these sponsors.

Author notes

Jenny Nordeen is an assistant professor in the Clinical Education Department, College of Chiropractic, University of Western States (8000 NE Tillamook Street, Portland, OR 97213; jnordeen@uws.edu). Christopher Browne is a professor and Director of Human Nutrition and Functional Medicine, College of Graduate Studies, University of Western States (8000 NE Tillamook Street, Portland, OR 97213; chbrowne@uws.edu). Brent Marshall is an assistant professor in sports medicine, College of Graduate Studies, University of Western States (8000 NE Tillamook Street, Portland, OR 97213; bmarshall@uws.edu).

Concept development: CAW. Design: CAW. Supervision: CAW. Data collection/processing: CAW, JN. Analysis/interpretation: CAW, JN, CB, BM. Literature search: CAW, BM. Writing: CAW. Critical review: CAW, JN, CB, BM.