Use of a Mobile Health (mHealth) Platform for Remote Assessment of Suicidal Ideation, Depression, and Anxiety: A Longitudinal Retrospective Study

In the United States, 1 in 5 adults suffer from a mental health disorder. The lifetime prevalence of anxiety and depression are 20.6% and 30.1%, respectively, and are both known to increase the likelihood of suicidal thinking and suicide.^[1–7] Suicide is the 10th leading cause of death, steadily increasing in occurrence, with 1.4 million suicide attempts in 2018, one in three of which were completed (488,344 deaths).^[8–10] Despite the high morbidity and mortality of suicide, depression, and anxiety in the United States, approximately 35% of adults with depression and 63% of adults with anxiety do not receive treatment.^[11,12] For those who do receive treatment, 54% of those with depression and 62% of those with anxiety do not receive adequate care.^[13]

There are internal and external reasons for undertreatment. Among internal factors are the internalized stigma of behavioral health conditions, lack of cultural sensitivity embedded within the intervention,^[14,15] the nature of the symptoms in and of themselves (e.g., apathy, hopelessness,^[16,17] and fear), and distrust of treatment options, such as medication.^[18,19] All these internal factors serve as barriers that reduce an individuals' desire to seek care. External factors include a shortage and uneven distribution of behavioral health care clinicians,^[20,21] high out-of-pocket costs,^[22,23] and difficulty with paying for medications.^[24]

Behavioral health focuses on the connections between how behaviors impact someone's physical and mental health. Behavioral health care clinicians work to counsel and support patients with mental illnesses.^[25] Among efforts to better address suicide, depression, and anxiety is the option of integrating behavioral health care into primary care. In the collaborative care model (CoCM),^[26] the primary care physician (PCP) is responsible for screening, diagnosis, and care of patients' behavioral health problems. In this model, the PCP receives support from a behavioral health care specialist who is available to treat individuals determined to need a higher level of care, such as providing psychotherapy or cognitive behavioral therapy (CBT). The integration of behavioral health care into primary care in a collaborative manner has proven effective for people of all ages,^[27–30] and has the potential for markedly improving access to behavioral health care. Improved access can then lead to improved outcomes for anxiety, depression, and suicide.^[31–33]

An important aspect of CoCM is measurement-based care (MBC), which can be defined as the practice of basing clinical care on client data collected throughout treatment.^[34] In MBC, structured patient-centered and validated measures of mental health symptoms, such as the patient health questionnaire-9 (PHQ-9) and generalized anxiety disorders-7 (GAD-7),^[35–37] are collected at regular intervals and used to adjust care plans as needed to improve client outcomes.^[38] Multiple meta-analyses show that MBC significantly improves behavioral health outcomes^[39–41] resulting in recent calls to scale MBC.^[42,43] These calls reflect the reality that despite the strong evidence supporting MBC, this approach is used by fewer than 20% of behavioral health practitioners.

Research shows widespread adoption has been limited by concerns about privacy, compliance with the HIPAA (Healthcare Insurance Portability and Accountability Act), and time to obtain and store paper/pen(cil) measurements at the individual patient/provider level.^[42] Limited use has also been attributed to the idea that the most at-risk may not charge their phones or may not be motivated to check in with an online service.^[44,45] Although MBC is used in some integrated behavioral health settings, there is little standardization of the delivery of assessments, treatment, and follow-up protocols and limited data on frequency of use.^[46]

Digital tools represent one platform to scale solutions and improve accessibility to MBC. Retention rates vary amongst technologies because there are no standardized measures;^[47] however, preliminary results from mobile health (mHealth) technology developed by Happify found a retention of 20% based on users completing a second PHQ-9 after 8 weeks.^[48] In a mHealth platform developed by the Veterans Affairs, 41.6% of users returned to use the app 1 month after downloading, but only 28.6% after 3 months.^[49] Some potential advantages of using smartphone apps for measurement of suicidal ideation, depression, and anxiety symptoms include that ability to utilize ecological momentary assessment (EMA), which is the real-time and real-world measurement of symptoms. The ability for patients to answer surveys at their convenience on their own device in their real-world environment may also decrease the Hawthorne effect (the tendency to behave or answer differently when under observation). A small prospective study demonstrated a strong positive correlation (r = 0.84) between smartphone-administered and traditional paper/pen(cil) PHQ-9 assessment.^[50] When data are collected in a technology platform, it may be possible to evaluate data from much larger patient groups and aggregate patient data for the purpose of improving behavioral health care treatment and outcomes at both the individual and population level.

In this study, we sought to determine whether changes in suicidal ideation and symptoms of anxiety and depression could be observed from baseline to 8 and 24 weeks based on use of an mHealth technology platform. We analyzed whether symptoms of suicidality, depression, and anxiety could be effectively captured and if these symptoms changed over time controlling for age, sex, and whether baseline symptoms were moderate or severe. We expected that it would be feasible to collect data consistent with an MBC approach via an mHealth platform.

We conducted an observational retrospective review of deidentified patient records with the aim of identifying whether measurements of suicidal ideation and symptoms of depression and anxiety changed among individuals receiving mHealth-supported integrated MBC. On behalf of the authors, Advarra institutional review board determined that this research was exempt from institutional review board ethical oversight per the Department of Health and Human Services regulation 45 CFR 46.104(d)(4).

In this study, all measurements were collected on the NeuroFlow mHealth platform, which was designed to allow patients to complete data on their own devices. All measurements were securely stored in a HIPAA-compliant database. In the platform, a clinician only accesses their patients' information via a digital dashboard that integrates with the electronic medical record (EMR). The reporting system in the mHealth platform alerts clinicians to patients who are not progressing or who have declining behavioral health. Features of the mobile app include daily self-rating scales for stress, mood, sleep, and pain; mindfulness tools, general health education, and step tracking. The mHealth platform is aligned with the recovery model through the use of trackers and wellness content and by also being complementary to psychiatric care.^[51,52] Among all mHealth app usage, 30-day user retention was 70% compared with a typical 30-day retention rate of 3.3% for behavioral health mobile apps.^[48]

Individuals' deidentified records were included in this observational retrospective review if they were an adult between 18 and 64, and had also completed at least one mHealth-measured PHQ-9 or GAD-7 with a score of 10 or higher, indicating moderate to severe depression, between May 2018 and May 2020. The presence of suicidal ideation was defined as a score of 1 or more on Q9 of the PHQ-9 and frequency of suicidal ideation was defined as the total score for that question. Individuals' records were included if a second PHQ-9 or GAD-7 score was available approximately 8 weeks later (56 ± 7 days), and the individual had used the mHealth app during at least 4 weeks of that 8-week period. Those who had a third measure at approximately 24 weeks (168 ± 7 days) were included in the 24-week sample if they used the mHealth app during at least 12 of the 24 weeks. A 14-day time window was used because mHealth users are prompted to take a survey 7 days before the due date and may take it until 7 days after.

We gathered age, sex, dates, and scores of PHQ-9 and GAD-7 assessments, and the cumulative number of mHealth-platform activities completed at the time of the final test. Information on socioeconomic factors including income level, health insurance type, and the social constructs of race and ethnicity were not gathered because these data are kept by the provider and were not in the deidentified mHealth records.

The PHQ-9 is among the briefest and most easily administered rating scales for depression. Patients self-rate the presence of symptoms in the previous 2 weeks on a scale of 0 (not at all) to 4 (nearly every day). Scores range from minimal (0–4), to mild (5–9), moderate (10–14), moderately severe (15–19), and severe depression (20–27).^[36] Question 9 (Q9) asks specifically about suicidal ideation and has been established as a predicter risk of suicide attempt and death by suicide.^[3,4] The GAD-7 is a 7-item questionnaire with 89% sensitivity and 92% specificity for generalized anxiety disorder. On the GAD-7, each question is rated by the patient on a scale of 0 (not present) to 3 (present almost every day).^[36] Clinically significant scores range from 5 to 9 for mild, 10 to 14 for moderate, and 15 to 21 for severe anxiety. The presence of anxiety disorders has been shown to predict increased risk of suicidal attempts.^[1,2,53]

The data retrieved were sent to an independent consultant who performed the statistical analysis. The proportion of people with suicidal ideation was calculated as the number of records with a score of 1 or more on Q9 of the PHQ-9 at each time point divided by the total number of records reviewed for that time point. For records in which an individual continued use for 24 weeks and completed a PHQ-9 at baseline, 8 weeks, and 24 weeks, the mean score on Q9 was considered to reflect the frequency of suicidal ideation.

For both the PHQ-9 and GAD-7, first scores and final scores were averaged to arrive at a mean baseline and mean final score on each measure for those who used the mHealth platform for 8 weeks, and the subset who continued use for 24 weeks. For each group, separate means were calculated for the following: (1) those who had moderately severe (PHQ-9 > 15) or severe (PHQ-9 > 20; GAD 7 > 15) depression or anxiety symptoms at baseline; and (2) those who had moderate (PHQ-9 = 10–14; GAD-7 = 10–14) symptoms at baseline. Student's t-tests were used to identify both the presence and quality of a difference between PHQ-9 and GAD-7 scores, as compared with baseline. A Student's t-test was employed given the small sample size. We used a prespecified 2-tailed paired Student's t-test to assess differences between mean baseline and final scores to evaluate for change in either direction. We used a paired test because both scores were available for every individual included in the study, creating a matched sample for the scores being compared.

We also carried out a prespecified analysis to evaluate whether any observed differences were affected by sex, age, or activity level by quartile after calculating the mean baseline and final scores for each sex, 3 different age ranges, and activity quartile. A prespecified 2-tailed unpaired Student's t-test was used to evaluate differences among these samples of different sizes and variations. A post-hoc analysis of activity level by age was conducted by merging and deduplicating all records reviewed for each measure and a 2-tailed unpaired Student's t-test was used to evaluate any differences in mean activity level by age. Results of the change in PHQ-9 and GAD-7 were analyzed was performed using Microsoft Excel on Windows 10. Results identifying age and suicidal ideation were analyzed using open source software: Python 3.7.0 (python.org, Python Software Foundation), Pandas 1.2.1 (pandas.pydata.org), NumPy 1.19.2 (numpy.org), and SciPy 1.6.0 (scipy.org). Figures were created in Matplotlib 3.3.2 (matplotlib.org).

We identified 4040 records in the mHealth platform database with a first-measured PHQ-9 score of 10 or more, suggesting moderate to severe depression, and 2783 records with a first GAD-7 score of 10 or more, suggesting moderate to severe anxiety. Comorbid depression and anxiety were present in approximately 27% of the records (1813/6823). We identified a subset of 1258 records with second scores for at least 1 of 2 measures recorded within 49 to 63 days of the first score (defined as the 8-week timepoint). Of these, 392 records had a third score recorded within 149 to 163 days of the first (defined as the 24-week timepoint). The rate of comorbidity in this subset was 24% (306/1258), similar to that of the overall record pool. There were 764 records with PHQ-9 scores at 8 weeks, and 185 of those had a 24-week score available. There were 797 records with an 8-week GAD-7 score, and 278 of those had a 24-week score available.

The only demographics maintained in the mHealth platform are age and sex (Table 1). All other demographics are kept by the provider/system the mHealth platform supports in providing MBC and were not included in the deidentified data.

We evaluated mean activity in the app over the 8-week period by age groups defined by decade of life (Fig. 1). Individuals under 20 years of age did more activities in the mHealth app compared with individuals over age 20 years, although this was statistically significant only for those with anxiety (p < 0.05), and the sample size for this group was small (n = 13). Individuals less than age 20 did more activities in the mHealth platform over the approximate 8 weeks at a mean 225 (SE, 38; p < 0.05 for all comparisons) for both those with anxiety (171 activities, SE, 30, p > 0.21) and for those with depression (p > 0.05).

People aged 20 to 59 years did a mean 99 to 137 activities (or 12–17/week) with no statistically significant differences seen (p > 0.06 for all comparisons).

Individuals aged 60 or older completed fewer activities (mean of 87 activities), but this reached statistical significance only in those with depression compared with individuals aged 30 to 39 or 50 to 59 with depression (p = 0.03). Mean activity levels for those with anxiety versus depression were similar in people under age 40 (p > 0.16), but after age 40, people with anxiety did significantly fewer activities compared with those with depression (p < 0.04). For those who had scores recorded at 24 weeks, the number of activities remained consistent over time (p > 0.12).

Change in the number of individuals endorsing or not endorsing suicidal ideation and the proportions with decreased scores on Q9 are shown in Figure 2A. Of the initial 764 records with PHQ-9 scores at 0 and 8 weeks, 25.0% (191/764) had scores > 0 on Q9. After 8 weeks of mHealth supported integrated behavioral health care, 14.7% (112/764; p < 0.001) had scores > 0. Of the subset (n = 185) who continued use for 24 weeks, 27.6% (51/185) had suicidal ideation at 8 weeks that was reduced to 21.7% (40/185; p < 0.05) at 24 weeks. Among 140 records with use for 8 weeks and scores > 0 on Q9 at baseline, the mean score decreased by 0.72 over the 8 weeks (p < 0.001). For the records with 24 weeks of use and > 0 on Q9 at baseline, there was no statistical difference from the larger group at weeks 0 and 8, and scores on Q9 dropped by a similar 0.74 over the first 8 weeks (p < 0.001) drop was similar. From weeks 8 to 24, this group had an increase of 0.27 in the mean score on Q9, although this was not statistically significant (p > 0.05).

Clinically significant reductions in depression and anxiety symptoms were seen over 8 weeks of mHealth-supported MBC, and improvements continued for those who had mHealth-supported MBC for 24 weeks (Figs. 3 and 4). Mean baseline PHQ-9 and GAD-7 scores were 14.69 (n = 764; standard error [SE], 4.09) and 14.30 (n = 797; SE, 3.31), respectively. At 8 weeks, mean PHQ-9 and GAD-7 scores were reduced to 10.50 (SE, 4.09; p < 0.001) and 10.10 (SE, 5.55; p < 0.001), respectively, just above the cutoff for mild depression or anxiety. For the subsets of people with depression (n = 185) or anxiety (n = 278) who had mHealth-supported MBC for 24 weeks, PHQ-9 and GAD-7 mean scores further decreased to 9.03 (SE, 7.09; p < 0.01) and 7.48 (SE, 6.54; p < 0.001), both indicative of mild depression or anxiety. Clinically significant improvements occurred whether people had severe anxiety (Figure 3B), severe or moderately severe depression (Figure 4B), or moderate depression or anxiety (Figs. 3C and 4C). There were no statistically significant differences among mean scores for males versus females or for different age groups in baseline, 8-week, or 24-week scores (p > 0.10 all comparisons).

Findings from this study indicate that it was possible to collect mental health data via an mHealth technology platform at scale. These results are promising given the importance of capturing symptom data in real time so treatment providers can make informed decisions about intervention strategies. Results also revealed that in patients with severe, moderately severe, or moderate depression or anxiety, clinically significant improvements were observed by using mHealth-supported MBC.

In this retrospective record review, we have shown that measurements can be obtained from large numbers of patients who remain engaged with the mHealth platform for up to 6 months. Of note, these measurements can also be used to observe changes in suicidal ideation and symptoms of depression and anxiety, which is important for implementing a key component of MBC, which is to adjust treatment in response to a patient's symptom improvement or deterioration. In addition, we found that overall symptoms of depression and anxiety decreased with mHealth-supported care, as did the presence and frequency of suicidal ideation as measured by Q9 on the PHQ-9. Frequency of suicidal ideation, however, did not change significantly from weeks 8 to 24, which may reflect the small size of the subgroup that continued use for 24 weeks. However, this lack of improvement could also reflect a higher likelihood that individuals who continue use of the platform past 8 weeks have suicidal ideation that does not, or is slower to, respond to treatment. Prospective studies are needed to further explore this possibility.

Increasing and improving the use of MBC in integrated behavioral health care is proposed as a means of addressing existing treatment gaps that result in close to two thirds of people with depression and anxiety not receiving best-practice care in the United States ^[29,30,31]. Measuring response to treatment is a critical part of helping individuals with depression and anxiety return to functional levels. Improvement to mild symptoms within 6 months of collaborative care (similar to what was seen in our record review of mHealth-supported MBC), correlates with an odds ratio of 12.3 for return to functional levels.^[54] Of note, improvements were seen in both severe and moderate levels of anxiety and depression, suggesting that the benefits of mHealth-supported MBC in collaborative care may be helpful for people who are typically more likely to have treatment-resistant depression and anxiety.^[55] Moreover, we saw response to treatment occurring within an 8-week period, supporting that response to treatment occurs more quickly with MBC, which was 5.6 weeks as compared with 11.3 weeks in a randomized clinical trial.^[56]

With social distancing in response to the COVID-19 pandemic and a resulting shift to telehealth for most behavioral health care services, the opportunity to have patients remotely self-report suicidal ideation and symptoms of depression and anxiety in real-time, real-world settings provides many advantages. Moreover, recent literature has shown that underserved populations are ready and willing to use mHealth technology for care, as the majority of this population does not have a primary care physician and are more willing to engage with their care in a remote fashion.^[57]

This study demonstrated that the mHealth platform has the potential to rapidly scale up the use of MBC within integrated behavioral health by overcoming the barriers of time for measurement, the Hawthorne effect, and concerns about confidentiality and HIPAA compliance. The mHealth platform leverages the recovery model to help and encourage users to stay in control of their lives^[58] but also support individuals in the event they do return to a premorbid state. Digital platforms can help engage patients, provide ease of measurement, and enable physicians to receive data without having to devote clinical or staff time to obtaining the measurements. While it may be claimed to be difficult to deliver assessments and measurements outside of the clinic,^[59,60] the NeuroFlow mHealth platform was created to lessen the burden on patient and provider. We observed high levels of engagement among mHealth platform users in this study, in which people of all ages used the app more than a dozen times per week.

As a retrospective record review of only deidentified information recorded in the mHealth database, this study was not able to determine what factors other than receiving mHealth-supported integrated MBC contributed to self-reported reductions in symptoms. Questions regarding effects of pharmacotherapy and specific neuropsychological modalities within the digitally supported collaborative approach remain to be answered. Information on socioeconomic status, comorbidities, and social determinants that may have affected outcomes also was not available. All retrospective studies are limited by definition as post hoc studies. We worked to minimize these limitations via use of validated measurement scales and independent data review and analysis. However, it should be noted that the NeuroFlow platform was developed with a strong focus on user-centered design and uses engagement tactics including behavioral economics. Future studies should be performed on other platforms that use remote MBC to analyze for any generalizability. Despite these limitations, finding a sizable group of patients, selected only for a specific duration of app use, had clinically significant improvements that improve the odds of remission and recovery supports use of mHealth-supported integrated MBC.

This study lays the foundation for mHealth-supported MBC, showcasing that digital technology platforms can effectively capture data in a manner similar to manual pen-and-paper or in-person through the EMR. Our research supports the use of digital methods to gather measurements that are essential to improved care for depression and anxiety, which will reduce risks for suicide. Changes in symptoms, including suicidal ideation, can be reliably assessed with the mHealth platform. Remote monitoring may offer a cost-effective solution to deliver initial assessment of self-harm and suicide at scale of a large practice. Further research to understand how clinicians use information generated by the mHealth platform in the care they provide is needed, in addition to the patient experience of the mHealth platform. Additional research may consist of understanding how and to what extent crisis resources are used within the app, and how this might improve patient outcomes.

We acknowledge the support of VT4 (Philadelphia, PA), an independent consulting firm, for providing statistical analysis and review of the data and the manuscript in exchange for consulting fees.