2021/12/09 by Hannah Metzler, H. Metzler, Hubert Baginski +7 · 3 citations
Computer Science · Medicine · Psychology · #Artificial intelligence #Benchmark (surveying) #Classifier (UML) #Computation and Language (cs.CL) #Computer science #Deep learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine learning #Medicine #Mental Health via Writing #Natural language processing #Poison control #Psychology #Social media #Suicidal ideation #Suicide and Self-Harm Studies #Suicide prevention #Support vector machine #World Wide Web #cs.CL #cs.LG
paper · pdf · doi:10.48550/arxiv.2112.04796
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2021/12/09 · openalex created_date 2021/12/31 · arxiv created 2022/06/28 · arxiv updated 2022/06/29 · openalex updated_date 2026/08/04
Research shows that exposure to suicide-related news media content is associated with suicide rates, with some content characteristics likely having harmful and others potentially protective effects. Although good evidence exists for a few selected characteristics, systematic large scale investigations are missing in general, and in particular for social media data. We apply machine learning methods to classify large quantities of Twitter data according to a novel annotation scheme that distinguishes 12 categories of suicide-related tweets. We then trained a benchmark of machine learning models including a majority classifier, an approach based on word frequency (TF-IDF with a linear SVM) and two state-of-the-art deep learning models (BERT, XLNet). The two deep learning models achieved the best performance in two classification tasks: In the first task, we classified six main content categories, including personal stories about either suicidal ideation and attempts or coping, calls for action intending to spread either problem awareness or prevention-related information, reporting of suicide cases, and other tweets irrelevant to these categories. The deep learning models reached accuracy scores above 73% on average across the six categories, and F1-scores in between 0.70 and 0.85 for all but the suicidal ideation and attempts category (0.51-0.55). In the second task, separating tweets referring to actual suicide from off-topic tweets, they correctly labeled around 88% of tweets, with BERT achieving F1-scores of 0.93 and 0.74 for the two categories, respectively. These classification performances are comparable to the state-of-the-art on similar tasks. By making data labeling more efficient, this work has enabled large-scale investigations on harmful and protective associations of social media content with suicide rates and help-seeking behavior.