Comparison of patient testimonials of orthognathic surgery on YouTube and TikTok

With the growth of the Internet, social media platforms have emerged as vital tools for communication, information dissemination, and entertainment. Established platforms like Facebook and YouTube (YT) have maintained continuous and increasing usage, boasting over 1 billion users as of 2018.¹  These platforms are widely used across all age demographics.²  In contrast, newer platforms such as TikTok (TT), Instagram, and Snapchat tend to attract a younger audience. TT has experienced a significant 12% increase in usage since 2021 and now reaches one-third of US adults.²  As the user base of these platforms continues to expand, their applications on the Internet have proliferated, influencing various aspects of daily life and societal interactions. Use of social media by healthcare professionals has notably increased, serving various purposes such as peer communication and enhancing productivity.³  Practitioners in diverse fields leverage the widespread use of social media platforms as educational tools and to reach a wider audience of potential patients. During the COVID-19 pandemic and global lockdowns, both health care professionals and the public turned to social media for disseminating information, exploring innovative approaches to accessing medical care, and making informed decisions regarding medical and dental treatments.

In dentistry, orthognathic surgery plays a crucial role in correcting dentofacial deformities and malocclusion.⁴  The surgical procedure may involve repositioning the maxilla, mandible, or both, to achieve an optimal functional and esthetic outcome, impacting surrounding structures such as muscles, blood supply, and nerves. The procedure is complex and carries various potential complications, including infections, fractures of fixation devices, damage to nerves such as the inferior alveolar and facial nerves, excessive bleeding, dental injuries, open bite, and potential relapse.⁵  Given the scope of the surgery and the potential risks, patients considering orthognathic surgery often seek guidance to make well-informed decisions. Prospective patients frequently turn to testimonials to gain firsthand insights from individuals who have undergone this invasive procedure. The information conveyed in these testimonials can significantly influence their decision-making process and perception about orthognathic surgery. Social media platforms enable patients to access this information conveniently from home, whether from dental and medical professionals or from experiences shared by other patients.

While the Internet and social media have facilitated the dissemination of information, concerns regarding misinformation also exist, often due to the absence of stringent regulations on platform content. It is crucial to assess the credibility of sources and educate patients on how to navigate social media for health information. This education is vital to ensure informed decision-making and to safeguard the well-being of the general population.

While reviewing the available literature on social media and its relation to orthodontics, it was evident that authors of most studies focused on topics such as clear aligners and braces.^6,7  Literature concerning the applications of social media for orthognathic surgery was noticeably absent. It is valuable for practitioners to understand the impact of information on social media on patient decision-making regarding invasive procedures such as orthognathic surgery. It is also important to analyze information across multiple platforms that cater to different generations of potential patients. Therefore, TT and YT were selected for this study. The objective was to investigate and compare the educational value of the most popular orthognathic surgery patient testimonials between TT and YT and to analyze the emotional responses of viewers through sentiment analysis of video comments on both platforms.

A cross-sectional comparative study was conducted using publicly available patient testimonials on social media. Due to the nature of the data collected, it was determined that Institutional Review Board approval was not required, as it did not meet the criteria for human subject research.

A preliminary search was conducted on TT and YT to identify search phrases likely to yield the most relevant videos. Two generic search phrases, “my jaw surgery” and “jaw surgery patient experience,” were used. Five videos were selected from each phrase based on the following exclusion criteria: videos in languages other than English, testimonials from nonpatients, videos with fewer than 5000 views, videos with poor audiovisual quality, videos without comments, videos lacking audible or written personal testimonials, and videos on irrelevant topics. Hashtags that were included in the caption of the TT videos were compiled along with the original two search phrases. The hashtags were then used as search phrases on TT. The first 30 videos from each hashtag search were analyzed, and those that did not meet the predetermined exclusion criteria were tallied. Each search phrase was evaluated based on the ratio of videos meeting the criteria to total videos searched. The minimum acceptable ratio on TT was set at 1:3, resulting in the selection of four search phrases. These four search phrases were subsequently searched on YT, and the same ratio evaluation process was applied. The minimum acceptable ratio on YT was determined to be 1:2. Ultimately, the search phrases “jaw surgery patient experience,” “jaw surgery recovery,” and “double jaw surgery journey” met the criteria with ratios of 1:3 on TT and 1:2 on YT and were, therefore, selected for the study.

A new account was established on TT to mitigate potential algorithm biases associated with personal accounts. Before initiating searches on YT, browser history and cookies were cleared, and the browser was set to private mode. Separate searches were conducted on TT and YT, with 30 videos collected under each of the selected search phrases on each platform, adhering to predefined exclusion criteria. The search was limited to the first 30 videos per search phrase based on previous research in which authors indicated that a significant majority of users limit their searches to the initial 20–30 videos.⁸  Videos were evaluated simultaneously by two examiners (L.R. and A.A.) using the established exclusion criteria, resulting in a total of 360 videos gathered for analysis. Video collection and relevant data (such as views, likes, comments, and video URLs) were spread over several days. One week was dedicated to gathering videos on TT, with each search phrase investigated on separate days. Similarly, another week was allocated for gathering videos on YT. Analysis of TT videos occurred in the subsequent week, followed by analysis of YT videos the week after. The second set of searches followed the same collection and analysis pattern immediately after concluding the first. Concurrently, the initial 30% of comments under each video were collected to facilitate sentiment analysis of audience responses.

To provide a qualitative assessment of the selected videos, an Information Completeness Score (ICS) was implemented. This score was based on specific criteria identified by the examiners to ensure potential patients received comprehensive information about the experience of orthognathic surgery.⁷  Each video and its accompanying caption were evaluated by both examiners, who awarded 1 point for the inclusion of each of the following topics: treatment procedure details, complications of treatment, comparison with other treatment modalities, cost of treatment, and use of visual aids. The total points awarded determined the video’s ICS score, ranging from 1 to 5. Due to the nature of social media and the deletion of some videos on TT prior to analysis, videos that were no longer accessible were assigned an ICS score of 0.

Subsequently, comments were analyzed to gauge viewer opinions on the patient testimonials. To ensure accurate analysis, a specific set of exclusion criteria was established. Comments meeting any of the following criteria were excluded: advertisements, containing only emoticons, not in English, posted by the video creator, replies to other comments, and comments deemed unrelated to viewer sentiment or inappropriate. After filtering, the comments were processed using Azure Machine Learning software, an add-on to Microsoft Excel (Microsoft Excel, Redmond, Wash). This software categorized each comment as positive, negative, or neutral and assigned a Sentiment Analysis Score (SAS) ranging from 0 to 1. A score of 0 indicated a negative sentiment, 0.5 denoted a neutral sentiment, and 1 signified a positive sentiment.⁹ 

Statistical analyses were completed using SPSS version 28 (IBM Corporation, Arm, NY) across 360 responses. Descriptive statistics were examined for views, likes, comments, ICS, and SAS across platforms, phrase groups and search groups. Kolmogorov-Smirnov tests determined that the data were not normally distributed. As a result, nonparametric Mann-Whitney U-tests were carried out to determine significant differences in ICS and SAS between platform groups. Independent-samples Kruskal-Wallis test with Bonferroni adjusted post hoc comparisons were conducted across phrase groups. Finally, Pearson correlations were carried out to evaluate the relationship between views, likes, comments, ICS, and SAS. Statistical significance was set at P < .05.

TT had a higher median number of views and likes (Table 1) than YT. On the other hand, YT had a higher median number of comments and ICS, while both platforms had the same median for SAS. ICS was significantly higher for YT than for TT (P < .001, Table 1; Figure 1) with YT having a median of 3, while TT had a median of 2 (Table 1). YT obtained the highest ICS score several times. No statistically significant difference was found for ICS among the search phrases on either platform. Mean and median SAS scores were similar for both platforms and were both in the neutral range (Table 1). No statistically significant difference was identified when evaluating SAS between the platforms or across phrase groups (Table 2). In addition, no relationship was observed between views, likes, comments, and SAS or ICS (Table 3).

With the rise in social media usage by the general population, people are increasingly seeking medical and dental information on these platforms. Patients considering procedures, particularly more invasive ones such as orthognathic surgery, often look to those who have undergone these treatments for insights into the patient experience. Patient testimonials posted on social media sites serve as a valuable source of information, facilitating dialog between patients and their health care providers. In this study, we aimed to assess the quality of content on two popular social media platforms and evaluate their impact on viewers, thereby determining which platforms engage a larger audience, how the content is perceived, and how this information can benefit potential patients. We are the first to analyze, compare, and classify the testimonials and emotional responses of surgical-orthodontic patients who will undergo orthognathic surgery on social media.

YT was found to have a significantly higher ICS than TT (P < .05, Table 1), suggesting that videos on YT provided more comprehensive information about the patient experience. This detailed content had the potential to influence patient decisions regarding whether to proceed with the procedure. Many YT videos delved into significant side effects such as prolonged swelling, facial numbness, infections necessitating urgent care, as well as dissatisfaction with treatment outcomes such as midline discrepancy or relapse requiring additional surgical intervention. Discussing the cost of the procedure was noticeably emphasized on YT compared with TT, which positively contributed to YT’s ICS. An intriguing observation was that YT videos also addressed the psychological challenges associated with orthognathic surgery, particularly concerning self-image and confidence. Orthognathic surgery can profoundly alter facial appearance, leading many patients to experience a sense of identity crisis.¹⁰  Such information is crucial for patients to consider when deciding whether to undergo orthognathic surgery. Therefore, it is helpful for patients to seek testimonials on platforms that offer comprehensive content to aid in their decision-making process. Higher ICS on YT could be attributed to the absence of time limitations,¹¹  allowing content creators more freedom to elaborate on their testimonials and cover a wider range of topics. In contrast, TT imposes a 3-minute limit on content, which compels creators to present their information more concisely.

Although YT featured more comprehensive testimonials, TT demonstrated greater audience engagement, evidenced by its higher median number of views (65,200 compared with YT’s 22,000) and likes (1925 compared with YT’s 261; Table 1). This indicates that TT testimonials reach a broader audience despite providing less detailed information. Given that orthognathic surgery is a highly invasive treatment with a demanding recovery process, significant commitment and thorough understanding are required among potential patients. TT has experienced substantial growth in popularity over the past 4 years and has become a preferred platform for many Americans.²  This preference may be influenced by TT’s time limitation, as shorter videos typically garner more viewer engagement.¹¹  TT has emerged as a swift and accessible source for various types of information; however, caution is advised when seeking medical information on this platform.

No significant differences between platforms were found for the other metrics. Each platform exhibited similar SAS across their testimonials (Table 1). This could indicate that, regardless of the completeness of the testimonials, viewers maintained similar opinions about orthognathic surgery. Additionally, no relationship was found between views, likes, or comments to ICS or SAS (Table 3), suggesting that engagement may not be related to the completeness of the video but rather with the platform on which it is presented. Although no difference was found between search phrases and ICS or SAS, a trend seemed to occur, with the median number of views and likes being higher for the phrase “jaw surgery recovery” with 65,250 views and 1101.5 likes (Table 2). This indicates that the choice of words used in postings may impact engagement levels regardless of platform.

The findings of this study provide insights for practitioners regarding which social media sites patients use to obtain medical information and how these platforms can be effectively leveraged. Social media has proven to be a valuable tool for health care professionals to engage with patients and peers,³  and using the right words or phrases can enhance audience engagement. With the increasing popularity of TT, practitioners should use this platform to educate the public about orthognathic surgery and its impact as well as to verify the accuracy of information posted by others. Additionally, if patients are adamant about turning to social media for information, practitioners should recommend that they seek comprehensive testimonials on YT to assist in their decision-making process. Orthodontists can also leverage their own TT profiles to guide patients toward valuable YT testimonials by linking informative videos. In today’s digital age, where social media plays a pivotal role in public life, it is essential for dental providers to acquaint themselves with the resources these platforms offer.

As the study progressed, some limitations were identified. During the sentiment analysis phase, the software encountered challenges in detecting nuances of language, such as irony, slang, and sarcasm. Stefanis et al.,⁹  in a similar evaluation using the same tool, also noted these difficulties but acknowledged the significance of sentiment analysis and predictive models in an era heavily influenced by information disseminated across diverse social media platforms, also expressing that this is particularly relevant in contexts involving “infoepidemics, infodemiology, and infosurveillance.” Increasing the number of comments collected could have enhanced the determination of audience sentiment significance. A longer time frame for video collection and a greater number of videos gathered could have resulted in a larger sample size, potentially yielding more meaningful results. Additionally, some videos became inaccessible during subsequent analysis, highlighting the importance of conducting analysis concurrently with data collection to ensure accessibility and completeness. In addition, while some viewers seek out testimonials to educate themselves about orthognathic surgery as a potential treatment option, others may encounter the videos due to social media algorithms. These individuals may or may not intend to undergo this procedure. Given the widespread reach of these platforms, defining the objectives of the commenters in this study was not feasible.

Authors of further studies should explore and expand upon the relationship between social media and orthognathic surgery. Alternative methods for evaluating sentiment should be employed to account for nuances in human language that may be challenging for some artificial intelligence programs to detect. Including human analysis to verify SAS could enhance the accuracy of findings. Also, analyzing testimonials on additional platforms would provide a more comprehensive understanding of the information available to potential patients across various social media sites. Other platforms, such as Instagram and Facebook, rank among the top five platforms used by American adults.²  Authors of future studies could explore the correlation between video length and ICS to ascertain whether these variables influence each other. Another area of interest could be assessing the accuracy of information posted about orthognathic surgery, whether in the form of patient testimonials or informational posts, across different platforms. This would provide insights into which platforms offer the most reliable information for patients, enabling practitioners to effectively guide patient education.

Based on the results of this study on prospective surgical-orthodontic patients, the following conclusions can be drawn.

• YT exhibited higher-quality information within the patient testimonials posted, as evidenced by the higher ICS than TT.

• Patient testimonials on both platforms did not show a discernible positive or negative influence on potential patient opinions.

• Orthodontists should use their TT profiles to help direct patients toward meaningful YT testimonials by connecting them with informative videos.

This study was supported, in part, by the National Center for Advancing Translational Sciences of the National Institutes of Health under Grant No. UL1 TR001450. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.