Introduction

This study aims to assess the understandability, actionability, and quality of online resources for the self-management (SM) of bipolar spectrum disorders in adults.

Methods

An online search using Google, Bing, and Yahoo! search engines was conducted to identify resources for bipolar disorder. Those that were published in English, discussed at least 1 method directed at improving an SM task, and were within the first 25 nonadvertisement results for each search were included. Resources directed specifically at adolescents were excluded. Understandability and actionability of the online resources were evaluated using the Patient Education Materials Assessment Tool (PEMAT). Quality of the online resources was evaluated using the DISCERN instrument. The number of SM tasks each resource discussed was also evaluated. Overall mean appropriateness was calculated by averaging the percentage scores of understandability, actionability, and quality.

Results

Fifty-two resources were included. The mean sample scores were 8.4 (SD, 2.1; range, 2-13; maximum, 15) for understandability, 2.2 (SD, 1.2; range, 0-4; maximum, 5) for actionability, and 46.1 (SD, 8.9; range, 30-57; maximum, 75) for quality. The overall mean appropriateness percentage was 53.5% (SD, 11.7%; range, 18%-77%), with a goal of at least 70%. Included resources addressed a mean of 7.1 tasks (SD, 2.5; range, 1-14; maximum, 20).

Discussion

Most online resources for the SM of bipolar disorder scored poorly for understandability and actionability based on PEMAT scores and had low to moderate scores for quality using the DISCERN instrument. Future online resources should be designed with the goal of increasing appropriateness for patients.

According to the National Institute of Mental Health,1  approximately 7.2 million American adults (2.8%) are affected by bipolar disorder. Patients with bipolar disorder may have significant impairment physically, emotionally, socially, and in occupational functioning. Because of the nature of bipolar disorder, undertreated or untreated individuals typically have relapses, significant symptoms, and severe impairment.1  Up to 20% of individuals with bipolar disorder, most of whom are untreated, die by suicide.2,3  Approximately half of individuals with bipolar disorder are untreated in any given year.4  Common barriers to seeking and receiving effective treatment include social stigma, limited knowledge about mental illness, lack of perceived need for treatment, and health care system–level barriers, such as availability of providers or cost of treatment.5 

Self-management (SM) interventions attempt to minimize barriers to treatment, reducing the risk of relapse.6  Broadly, SM interventions empower patients with the information and skills necessary for the successful day-to-day management of their chronic condition when they are between visits with providers, therapists, and other health professionals. The interventions are intended to complement and be implemented in conjunction with more traditional treatment modalities, such as pharmacotherapy, psychotherapy, and lifestyle modifications. SM activities can be categorized into 3 types of tasks: focusing on illness needs, activating resources, and living with a chronic illness.7  Focusing on illness needs outlines the tasks and skills necessary for individuals to take care of their bodies and illness-specific concerns, such as taking all medications as prescribed and recognition of early symptoms of a mood episode.7  Activating resources covers maximizing support from individuals (eg, family members, friends, health care providers) and community resources and services (eg, spiritual resources, social and transportation services) to aid in optimal disease management.7  Lastly, living with a chronic illness covers the tasks and skills related to coping with the illness and growing as a person; processing emotions, adjusting to illness and new self, and integrating into daily life are potential examples.7  The benefits of implementing tasks within these 3 core SM areas include promoting active engagement of patients in their treatment and prevention of relapse; improved confidence with medical management, role management, and emotional management; and increased self-sufficiency and decreased patient reliance on health care professionals. These benefits culminate in improved quality of life, clinical outcomes, and functional outcomes for individuals with chronic illness.6-8  Therefore, SM interventions can and should be recommended to any patient with a chronic disease willing to implement them, regardless of disease state or severity.

Although SM resources are commonly given to patients by health care providers, the Internet represents a secondary source of readily accessible SM resources. The first step for most people seeking specific health information is to use search engines.9  However, the appropriateness, quality, and consistency of online health-related search results is questionable. Zhang et al10  found the overall quality of online health information was problematic and that online materials contained outdated or even harmful information. Rathod et al11  determined online resources for the SM of depression averaged a grade 10 reading level, exceeding both the ideal grade 5 reading level and the acceptable grade 8 reading level. Considering the potential benefits of implementing SM tasks in bipolar disorder and the significant number of patients seeking health information via online search engines, there is value in evaluating the appropriateness of online resources found by Internet searches for the SM of bipolar disorder.

The aim of this study is to assess the understandability, actionability, and quality of these online resources to determine their overall appropriateness. Understandability refers to when consumers of different backgrounds and varying levels of health literacy can process and explain key information.12  Actionability refers to the ability of these consumers to identify what they can do based on the information presented.12  Quality refers to resources being created using good evidence and aligning with current treatment guidelines and evidence-based practices.13 

This study identified online resources for the SM of bipolar disorder through an online search conducted in February 2020. Google, Bing, and Yahoo! search engines were used to identify up to 78 online resources—up to 2 from each completed search. A total of 13 search terms (Table 1) were used within each of the 3 search engines for a total of 39 completed searches. The search terms were modeled after Rathod et al,11  a similar study for online SM resources for depression which used search frequency data to determine search terms most likely to be used by patients. The goal of the study was to find resources patients would likely locate in their own searches. As such, the inclusion criteria for the study were lenient. Resources were eligible for inclusion if they discussed at least 1 method directed at improving a SM task (Table 2) and were published in English. Resources also had to be within the first 25 nonadvertisement search results for each search conducted, which mimicked typical search behavior of prioritizing early results and provided a defined cutoff to terminate the search when repeated results increased in prevalence. The first 2 nonadvertisement, nonrepeated resources for each search were included. Resources directed at adolescents with bipolar disorder were excluded. The primary researcher evaluated all identified resources to determine if they met inclusion criteria. The search was conducted from a central US Internet protocol (IP) address using the incognito mode of the Google Chrome Web browser. The browser history, cache, and cookies were cleared prior to the search to mitigate past searches imparting bias on the study search results.

TABLE 1

Used search terms

Used search terms
Used search terms
TABLE 2

Incidence of self-management tasks (n = 52)

Incidence of self-management tasks (n = 52)
Incidence of self-management tasks (n = 52)

All included online SM resources were evaluated by the primary researcher using the Patient Education Materials Assessment Tool (PEMAT), a validated instrument that measures both the understandability and the actionability of patient education resources.12  The PEMAT is available in 2 forms, PEMAT-Print and PEMAT-Audiovisual. Only the PEMAT-Print, which includes materials that can be printed or viewed from a Web site, was used for the evaluation of included resources. No formal training is required to use the PEMAT, although a detailed user's guide is available. The PEMAT includes a total of 19 questions for understandability and 7 questions for actionability.12  Answer options include Agree (1 point), Disagree (0 points), or Not Applicable (excluded when calculating total score, decreasing the total possible points).12  Four items in the PEMAT: Understandability domain were not included in the scoring because of a lack of presence in the resources: clear and easy to understand numbers, no required calculations, easy to hear audio, and simple tables with short and clear row and column headings. Likewise, 2 items in the PEMAT: Actionability domain were not included in the scoring: simple instructions or examples of how to perform calculations and explains how to use figures. Therefore, the PEMAT: Understandability domain scores were calculated out of 15 total possible points and the PEMAT: Actionability domain scores were calculated out of 5 total possible points. Both scores were summed, and a percentage was calculated. A resource was considered understandable and/or actionable if it scored ≥70% on either section, which was used as the goal result for the PEMAT in previous studies.11-13 

All included SM resources were also evaluated by the primary researcher using the DISCERN instrument, a validated tool that assesses the quality of patient education resources.14  No formal training is required to use the DISCERN instrument, although detailed guidance is available regarding the scoring of each question. The DISCERN instrument includes 16 items that assess the quality of materials in 3 sections. The first 8 items assess the reliability of the publication and trustworthiness of information. The next 7 items assess specific details of the information concerning treatment choices. The final item is an overall rating of the material and was omitted from this study to allow a more detailed breakdown of quality scores. Each item is rated from 1 to 5, with higher scores reflecting higher overall quality, for a maximum possible score of 75. Scores on items 1 to 15 were summed and a percentage calculated based on the total possible of 75. A resource was considered to be high quality with a score of ≥70%, which was used as the goal result for the DISCERN instrument in a previous study.11  Lastly, an overall mean appropriateness percentage for each resource was calculated by averaging the percentage scores of the 3 scales. This calculation was developed to give an assessment of the resource as a whole. Previous studies did not calculate composite scores for resources.

An SM task checklist consisting of 20 individual tasks split between 3 core areas (Table 2) was developed based on research by Schulman-Green et al.7  All resources were evaluated based on the number of SM tasks included. Other information, including publication location, publishing year, last updated year, publication entity (eg, private organization vs nonprofit entities), format (eg, Web page vs electronic booklet), and presence of advertisements, was also collected. All study data were collected and managed using REDCap electronic data capture tools hosted at the Center for Health Insights of the University of Missouri-Kansas City.15  Following data collection, data were exported to Microsoft Excel in order to calculate descriptive statistics for the 2 PEMAT domains and the DISCERN instrument.

The Internet search identified 52 resources (Table 3),16-67  with resources published from the United States (n = 39), United Kingdom (n = 5), Canada (n = 4), and Australia (n = 4). Only 52 resources were eligible for inclusion as repeat search results increased in prevalence, causing some searches to terminate after 25 nonadvertisement results with fewer than 2 resources identified. Most resources were created by private organizations (n = 31), with the remainder created by professional organizations (n = 3), universities (n = 1), hospital systems (n = 2), government organizations (n = 2), or other nonprofit entities (n = 13). Most of the resources were Web pages (n = 42). The rest were formatted as personal blogs (n = 7) or electronic booklets (n = 3). Approximately 30% of resources were missing a year of publication or a last updated year, and 6 resources were missing both. Most resources contained advertisements (n = 31) directed at readers.

TABLE 3

Characteristics of top and bottom 10 results included in final sample

Characteristics of top and bottom 10 results included in final sample
Characteristics of top and bottom 10 results included in final sample

The mean PEMAT: Understandability score was 8.4 (SD, 2.1; range, 2-13) out of a possible 15 points. Seven resources (13.5%) in the understandability domain included relevant visual aids and 13 resources (25%) included a summary of key points. The mean PEMAT: Actionability score was 2.2 (SD, 1.2; range, 0-4) out of a possible 5 points. Within the actionability domain, 21 resources (40.4%) broke at least 1 action down into manageable steps, and 12 resources (23.1%) included a tangible tool, such as a mood charting tool, to aid in completion of an action. The mean DISCERN instrument score assessing quality was 46.1 (SD, 8.9; range, 30-57) out of a possible 75 points. One quality concern noted was poor referencing of source materials. A full reference list was available for 10 resources (19.2%), in-text citations alone were used in 15 resources (28.8%), and no referencing information was found in 29 resources (55.7%). Additionally, 16 resources (30.8%) provided external sources of information beyond the publishing Web site. The overall mean appropriateness percentage was 53.5% (SD, 11.7%; range, 18%-77%). The Figure displays the appropriateness percentage for each of the 52 included resources. Four resources (7.7%) had an appropriateness score reaching or exceeding 70%. Of the 20 SM tasks, the reviewed resources addressed a mean of 7.1 tasks (SD, 2.5; range, 1-14). The most commonly included SM tasks were changing behaviors to minimize disease impact (n = 46), recognizing and managing body responses (n = 45), and obtaining and managing social support (n = 35). The least common were reevaluating life (n = 0), addressing social and environmental challenges (n = 1), and adjusting to new self (n = 3). The incidence of the other SM tasks is detailed in Table 2.

FIGURE

Overall percent appropriateness for included resources

FIGURE

Overall percent appropriateness for included resources

Close modal

Despite the potential benefits of implementing SM tasks in bipolar disorder and the prevalence of patients seeking health information via online search engines, this is the first study to evaluate the appropriateness of online SM resources for bipolar disorder. This study demonstrated that most easily accessible online SM resources for bipolar disorder scored poorly on understandability and actionability domains.

Understandability of these resources could be improved with consistent inclusion of short summaries of key takeaway points and relevant visual aids. Actionability of these resources could be improved if resources more consistently broke actions down into explicit, manageable steps and provided tangible tools such as mood charting templates.

Although DISCERN instrument quality scores were consistently the highest scoring aspect, many resources had characteristics indicative of low-quality resources. These included a low rate of referencing sources of information and a high rate of only providing additional sources of information within the hosting site. These deficiencies could easily be remedied by sourcing information used with in-text citations and by providing a variety of outside organizations and Web sites for patients seeking additional information.

This review differs from similar studies because of the decision to not include readability assessments. The applicability of readability assessments to educational materials is limited because many of the tools score readability based on either the number of letters or syllables of words in a sample of the publication. Educational materials for patients often require the repetition of certain long or complex words that are potentially vital for complete education. For example, in bipolar disorder materials, it is not uncommon to see words such as depression, hypomania, stabilization, and anticonvulsant. When appropriately explained, such terms can elevate educational resources rather than limit them. Additionally, similar studies have found a lack of correlation between readability levels and understandability scores as assessed by the PEMAT.11,13 

This study has several strengths, including a reasonably large sample size of resources (n = 52). Additionally, resources were identified using online search strategies designed to mirror those used by consumers, and the inclusion criteria were lenient. Lastly, validated tools such as the PEMAT and the DISCERN instrument were used to evaluate the resources. This study also has limitations. Using search engines to identify resources creates a bias toward popular sources of information rather than reputable or high-quality sources of information. Additionally, the search was conducted from the central United States; therefore, search results studied may be different from results from other areas of the country or world. Although both the PEMAT and the DISCERN instrument include in-depth scoring rubrics, many scoring items relied on subjective interpretation. Lastly, although the DISCERN instrument addresses sources of information, the accuracy of the information included in each resource was not evaluated directly.

Because of low scores for understandability and actionability and low to moderate scores for quality, the average overall appropriateness score of online resources for the self-management of bipolar disorder was below the goal of at least 70%. Online resources should be updated or designed with the goal of increasing understandability, actionability, and quality. The PEMAT domains and the DISCERN instrument questions could be used as a guide when designing these resources.

1. 
National Institute of Mental Health [Internet].
Bipolar disorder
[updated
2017
Nov]. Available from: https://www.nimh.nih.gov/health
2. 
Dome
P,
Rihmer
Z,
Gonda
X.
Suicide risk in bipolar disorder: a brief review
.
Medicina (Kaunas)
.
2019
;
55
(8)
:
403
.
DOI: 10.3390/medicina55080403. PubMed PMID: 31344941; PubMed Central PMCID: PMC6723289.
3. 
Latalova
K,
Kamaradova
D,
Prasko
J.
Suicide in bipolar disorder: a review
.
Psychiatr Danub
.
2014
;
26
(2)
:
108
-
14
.
PubMed PMID: 24909246.
4. 
Lingam
R,
Scott
J.
Treatment non-adherence in affective disorders
.
Acta Psychiatrica Scand
.
2002
;
105
(3)
:
164
-
72
.
5. 
Blixen
C,
Perzynski
AT,
Bukach
A,
Howland
M,
Sajatovic
M.
Patients' perceptions of barriers to self-managing bipolar disorder: a qualitative study
.
Int J Soc Psychiatry
.
2016
;
62
(7)
:
635
-
44
.
DOI: 10.1177/0020764016666572. PubMed PMID: 27647605; PubMed Central PMCID: PMC5578914.
6. 
Grady
PA,
Gough
LL.
Self-management: a comprehensive approach to management of chronic conditions
.
Am J Public Health
.
2014
;
104
(8)
:
e25
-
e31
.
DOI: 10.2105/AJPH.2014.302041. PubMed PMID: 24922170; PubMed Central PMCID: PMC4103232.
7. 
Schulman-Green
D,
Jaser
S,
Martin
F,
Alonzo
A,
Grey
M,
McCorkle
R,
et al
Processes of self-management in chronic illness
.
J Nurs Scholarsh
.
2012
;
44
(2)
:
136
-
44
.
DOI: 10.1111/j.1547-5069.2012.01444.x. PubMed PMID: 22551013; PubMed Central PMCID: PMC3366425.
8. 
Murray
G,
Suto
M,
Hole
R,
Hale
S,
Amari
E,
Michalak
EE.
Self-management strategies used by “high functioning” individuals with bipolar disorder: from research to clinical practice
.
Clin Psychol Psychother
.
2010
;
18
(2)
:
95
-
109
.
DOI: 10.1002/cpp.710. PubMed PMID: 20572206.
9. 
Eysenbach
G.
Health-related searches on the Internet
.
JAMA
.
2004
;
291
(24)
:
2946
.
10. 
Zhang
Y,
Sun
Y,
Xie
B.
Quality of health information for consumers on the web: a systematic review of indicators, criteria, tools, and evaluation results
.
J Assoc Inf Sci Technol
.
2015
;
66
(10)
:
2071
-
84
.
11. 
Rathod
P,
Ould Brahim
L,
Belzile
E,
Lambert
S.
An evaluation of the suitability, readability, and quality of online resources for the self-management of depression
.
Patient Educ Couns
.
2019
;
102
(5)
:
952
-
60
.
12. 
Agency for Health Research and Quality [Internet].
The Patient Education Materials Assessment Tool (PEMAT) and user's guide
[updated
2019
13. 
Lipari
M,
Berlie
H,
Saleh
Y,
Hang
P,
Moser
L.
Understandability, actionability, and readability of online patient education materials about diabetes mellitus
.
Am J Health Syst Pharm
.
2019
;
76
(3)
:
182
-
86
.
DOI: 10.1093/ajhp/zxy021. PubMed PMID: 31408087.
14. 
Radcliff Medical Press Ltd [Internet].
The DISCERN Instrument
[updated
1998
15. 
Harris
PA,
Taylor
R,
Thielke
R,
Payne
J,
Gonzalez
N,
Conde
JG.
Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support
.
J Biomed Inform
.
2009
;
42
(2)
:
377
-
81
.
DOI: 10.1016/j.jbi.2008.08.010. PubMed PMID: 18929686; PubMed Central PMCID: PMC2700030.
16. 
Smith
M,
Segal
J,
Segal
R.
Living with bipolar disorder
[updated
2019]
.
HelpGuide
[Internet].
17. 
Peisley
T.
Self-care for managing mania
[updated
2017
Mar].
SANE Australia
[Internet].
18. 
Roche
L.
Self help strategies for bipolar disorder: lifestyle changes and crisis plans
[updated
2015
Mar].
Mentalhealth Matters
[Internet].
19. 
Intermountain Healthcare. Bipolar disorder information for patients and families
[updated
2004]
.
Intermountain Healthcare
[Internet].
20. 
Anbar
R.
How to cope with bipolar disorder (manic depression)
[cited
2021
Jul].
WikiHow
[Internet].
21. 
Mind UK [Internet].
Bipolar disorder
[updated
2018]
.
22. 
Burgess
L.
What to know about bipolar disorder and anger
[updated
2019
Feb].
MedicalNewsToday
[Internet].
23. 
Healthwise Staff, Magellan Health [Internet].
Bipolar disorder
[cited
2021
24. 
The Royal Australian & New Zealand College of Psychiatrists.
Self-care for bipolar disorder
[updated
2017
Feb].
Your Health in Mind
[Internet].
25. 
International Bipolar Foundation [Internet].
Self-management techniques for bipolar disorder
[cited
2021
26. 
Association for Natural Psychology.
50 natural ways to manage and overcome symptoms of bipolar disorder
[updated
2014]
.
Association for Natural Psychology [Internet]
.
27. 
Fast
J.
Bipolar depression management tips
[updated
2019
Jul].
HealthyPlace
[Internet].
28. 
ULifeline [Internet].
Tips to successfully manage your bipolar disorder
[updated
2020]
.
29. 
Empty Next Lifestyle [Internet].
How to practice self-care when you have bipolar disorder
[updated
2019]
.
30. 
Heinz
C,
Prechter Bipolar Research Program. Living with bipolar disorder
[cited
2021
Jul].
University of Michigan
[Internet].
31. 
WebMD [Internet].
Depression in bipolar disorder: what you can do
[updated
2018]
.
32. 
MentalHelp [Internet].
Self-help for bipolar disorder
[cited
2021
33. 
PsychCentral [Internet].
Psychotherapy and self-help for bipolar disorder
[updated
2016]
.
34. 
Roche
M.
My story: bipolar disorder and self-care
[updated
2016
Mar].
SANE Australia
[Internet].
35. 
Marano
HE.
Managing bipolar disorder
[updated
2016
Jun].
Psychology Today
[Internet].
36. 
Burgess
L.
Is there a cure for bipolar disorder?
[updated
2019
Feb].
Medical News Today
[Internet].
37. 
Gillette
H.
Small steps you can take today to best bipolar disorder
[updated
2016]
.
PsychCentral
[Internet].
38. 
Worth
T.
10 tips for treating bipolar disorder
[updated
2012
Feb].
Health
[Internet].
39. 
The Recovery Village [Internet].
10 tips for overcoming bipolar disorder
[updated
2019]
.
40. 
ULifeline [Internet].
Tips to successfully manage your bipolar disorder
[updated
2020]
.
41. 
Tartakovsky
M.
The 4 keys to managing bipolar disorder
[updated
2016
May].
PsychCentral
[Internet].
42. 
Tracy
N.
Managing bipolar disorder: 5 things you must know
[updated
2019
Jun].
Healthy Place
[Internet].
43. 
Smith
K.
6 tips to live better with bipolar disorder
[updated
2019
Jul].
Psycom
[Internet].
44. 
Sheehan
J.
Coping with bipolar mood swings
[updated
2008
Nov].
Everyday Health
[Internet].
45. 
Casabianca
SS.
Self-help strategies for bipolar disorder
[updated
2016]
.
PsychCentral
[Internet].
46. 
Tartakovsky
M.
Living with bipolar disorder: what to expect
[updated
2016]
.
PsychCentral
[Internet].
47. 
National Health Service [Internet].
Living with bipolar disorder
[updated
2019
48. 
Kaplan
C.
4 Self-help tips for coping with bipolar disorder
[updated
2019]
.
New Life Outlook Bipolar
[Internet].
49. 
Tracy
N.
How to cope with bipolar disorder: bipolar coping skills
[updated
2019
Jun].
Healthy Place
[Internet].
50. 
Heinesh
B.
Coping with bipolar disorder--it is possible
[updated
2019
Mar].
New Life Outlook Bipolar
[Internet].
51. 
Reach Out [Internet].
Self-management for bipolar disorder
[cited
2021
52. 
National Alliance on Mental Illness [Internet].
Bipolar disorder
[updated
2017]
.
53. 
Mayo Clinic [Internet].
Bipolar disorder
[updated
2016]
.
54. 
Tartakovsky
M.
What are my bipolar disorder treatment options?
[updated
2016]
.
PsychCentral
[Internet].
55. 
National Institute of Mental Health [Internet].
Bipolar disorder
[updated
2020
56. 
Here to Help [Internet].
Self control and bipolar disorder
[updated
2003]
.
57. 
Tartakovsky
M.
The most effective therapies for treating bipolar disorder
[updated
2019
May].
PsychCentral
[Internet].
58. 
Bipolar Lives [Internet].
Bipolar wellness plan
[cited
2021
59. 
Archer
A.
A self-help guide to living with bipolar
[updated
2015
Jun].
Good Therapy
[Internet].
60. 
Michalak
E.
Self-care and bipolar disorder: putting yourself in the driver's seat
[updated
2017
Dec].
CREST.BD
[Internet].
61. 
Tracy
N.
Realistic self-care in bipolar disorder
[updated
2018
Jul].
Healthy Place [Internet]. Web.
62. 
Fleener
P.
Bipolar disorder self care
[cited
2021
Jul].
Mental Health Today
[Internet].
63. 
Oliver
D.
Bipolar disorder self-care
[updated
2010]
.
Bipolar Central
[Internet].
64. 
US News [Internet].
A patient's guide to bipolar disorder
[cited
2021
65. 
Casarella
J.
Everyday tips for living with bipolar disorder
[updated
2018]
.
WebMD
[Internet].
66. 
Mind UK [Internet].
Understanding bipolar disorder
[updated
2015]
.
67. 
Hvilivitzky
T.
Top 10 ways to cope with bipolar mood swings
[updated
2019]
.
BP Hope
[Internet].

Competing Interests

Disclosures: The authors do not have a financial interest/arrangement or affiliation with 1 or more organizations that could be perceived as a real or apparent conflict of interest in the context of the subject of this study.

This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial 3.0 License, which permits non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.