{"675889":{"#nid":"675889","#data":{"type":"news","title":"Study Highlights Challenges in Detecting Violent Speech Aimed at Asian Communities","body":[{"value":"\u003Cp\u003EA research group is calling for internet and social media moderators to strengthen their detection and intervention protocols for violent speech.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003ETheir study of language detection software found that algorithms struggle to differentiate anti-Asian violence-provoking speech from general hate speech. Left unchecked, threats of violence online can go unnoticed and turn into real-world attacks.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EResearchers from Georgia Tech and the Anti-Defamation League (ADL) teamed together\u0026nbsp;\u003Ca href=\u0022https:\/\/claws-lab.github.io\/violence-provoking-speech\/\u0022\u003E\u003Cstrong\u003Ein the study\u003C\/strong\u003E\u003C\/a\u003E. They made their discovery while testing natural language processing (NLP) models trained on data they crowdsourced from Asian communities.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u201cThe Covid-19 pandemic brought attention to how dangerous violence-provoking speech can be. There was a clear increase in reports of anti-Asian violence and hate crimes,\u201d said\u0026nbsp;\u003Ca href=\u0022https:\/\/gaurav22verma.github.io\/\u0022\u003E\u003Cstrong\u003EGaurav Verma\u003C\/strong\u003E\u003C\/a\u003E, a Georgia Tech Ph.D. candidate who led the study.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u201cSuch speech is often amplified on social platforms, which in turn fuels anti-Asian sentiments and attacks.\u201d\u003C\/p\u003E\u003Cp\u003EViolence-provoking speech differs from more commonly studied forms of harmful speech, like hate speech. While hate speech denigrates or insults a group, violence-provoking speech implicitly or explicitly encourages violence against targeted communities.\u003C\/p\u003E\u003Cp\u003EHumans can define and characterize violent speech as a subset of hateful speech. However, computer models struggle to tell the difference due to subtle cues and implications in language.\u003C\/p\u003E\u003Cp\u003EThe researchers tested five different NLP classifiers and analyzed their F1 score, which measures a model\u0027s performance. The classifiers reported a 0.89 score for detecting hate speech, while detecting violence-provoking speech was only 0.69. This contrast highlights the notable gap between these tools and their accuracy and reliability.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EThe study stresses the importance of developing more refined methods for detecting violence-provoking speech. Internet misinformation and inflammatory rhetoric escalate tensions that lead to real-world violence.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EThe Covid-19 pandemic exemplified how public health crises intensify this behavior, helping inspire the study. The group cited that anti-Asian crime across the U.S. increased by 339% in 2021 due to malicious content blaming Asians for the virus.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EThe researchers believe their findings show the effectiveness of community-centric approaches to problems dealing with harmful speech. These approaches would enable informed decision-making between policymakers, targeted communities, and developers of online platforms.\u003C\/p\u003E\u003Cp\u003EAlong with stronger models for detecting violence-provoking speech, the group discusses a direct solution: a tiered penalty system on online platforms. Tiered systems align penalties with severity of offenses, acting as both deterrent and intervention to different levels of harmful speech.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u201cWe believe that we cannot tackle a problem that affects a community without involving people who are directly impacted,\u201d said\u0026nbsp;\u003Ca href=\u0022https:\/\/jiaweizhou.me\/\u0022\u003E\u003Cstrong\u003EJiawei Zhou\u003C\/strong\u003E\u003C\/a\u003E, a Ph.D. student who studies human-centered computing at Georgia Tech.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u201cBy collaborating with experts and community members, we ensure our research builds on front-line efforts to combat violence-provoking speech while remaining rooted in real experiences and needs of the targeted community.\u201d\u003C\/p\u003E\u003Cp\u003EThe researchers trained their tested NLP classifiers on a dataset crowdsourced from a survey of 120 participants who self-identified as Asian community members. In the survey, the participants labeled 1,000 posts from X (formerly Twitter) as containing either violence-provoking speech, hateful speech, or neither.\u003C\/p\u003E\u003Cp\u003ESince characterizing violence-provoking speech is not universal, the researchers created a specialized codebook for survey participants. The participants studied the codebook before their survey and used an abridged version while labeling.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003ETo create the codebook, the group used an initial set of anti-Asian keywords to scan posts on X from January 2020 to February 2023. This tactic yielded 420,000 posts containing harmful, anti-Asian language.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EThe researchers then filtered the batch through new keywords and phrases. This refined the sample to 4,000 posts that potentially contained violence-provoking content. Keywords and phrases were added to the codebook while the filtered posts were used in the labeling survey.\u003C\/p\u003E\u003Cp\u003EThe team used discussion and pilot testing to validate its codebook. During trial testing, pilots labeled 100 Twitter posts to ensure the sound design of the Asian community survey. The group also sent the codebook to the ADL for review and incorporated the organization\u2019s feedback.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u201cOne of the major challenges in studying violence-provoking content online is effective data collection and funneling down because most platforms actively moderate and remove overtly hateful and violent material,\u201d said Tech alumnus\u0026nbsp;\u003Ca href=\u0022https:\/\/www.linkedin.com\/in\/rynaagrover\/\u0022\u003E\u003Cstrong\u003ERynaa Grover\u003C\/strong\u003E\u003C\/a\u003E (M.S. CS 2024).\u003C\/p\u003E\u003Cp\u003E\u201cTo address the complexities of this data, we developed an innovative pipeline that deals with the scale of this data in a community-aware manner.\u201d\u003C\/p\u003E\u003Cp\u003EEmphasis on community input extended into collaboration within Georgia Tech\u2019s College of Computing. Faculty members\u0026nbsp;\u003Ca href=\u0022https:\/\/faculty.cc.gatech.edu\/~srijan\/\u0022\u003E\u003Cstrong\u003ESrijan Kumar\u003C\/strong\u003E\u003C\/a\u003E and\u0026nbsp;\u003Ca href=\u0022http:\/\/www.munmund.net\/\u0022\u003E\u003Cstrong\u003EMunmun De Choudhury\u003C\/strong\u003E\u003C\/a\u003E oversaw the research that their students spearheaded.\u003C\/p\u003E\u003Cp\u003EKumar, an assistant professor in the School of Computational Science and Engineering, advises Verma and Grover. His expertise is in artificial intelligence, data mining, and online safety.\u003C\/p\u003E\u003Cp\u003EDe Choudhury is an associate professor in the School of Interactive Computing and advises Zhou. Their research connects societal mental health and social media interactions.\u003C\/p\u003E\u003Cp\u003EThe Georgia Tech researchers partnered with the ADL, a leading non-governmental organization that combats real-world hate and extremism. ADL researchers\u0026nbsp;\u003Ca href=\u0022https:\/\/binny-mathew.github.io\/\u0022\u003E\u003Cstrong\u003EBinny Mathew\u003C\/strong\u003E\u003C\/a\u003E and\u0026nbsp;\u003Ca href=\u0022http:\/\/www.jordankraemer.com\/\u0022\u003E\u003Cstrong\u003EJordan Kraemer\u003C\/strong\u003E\u003C\/a\u003E co-authored the paper.\u003C\/p\u003E\u003Cp\u003EThe group will present its paper at the\u0026nbsp;\u003Ca href=\u0022https:\/\/2024.aclweb.org\/\u0022\u003E\u003Cstrong\u003E62nd Annual Meeting of the Association for Computational Linguistics\u003C\/strong\u003E\u003C\/a\u003E (ACL 2024), which takes place in Bangkok, Thailand, Aug. 11-16\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EACL 2024 accepted 40 papers written by Georgia Tech researchers. Of the 12 Georgia Tech faculty who authored papers accepted at the conference, nine are from the College of Computing, including Kumar and De Choudhury.\u003C\/p\u003E\u003Cp\u003E\u201cIt is great to see that the peers and research community recognize the importance of community-centric work that provides grounded insights about the capabilities of leading language models,\u201d Verma said.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u201cWe hope the platform encourages more work that presents community-centered perspectives on important societal problems.\u201d\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cem\u003EVisit \u003C\/em\u003E\u003Ca href=\u0022https:\/\/sites.gatech.edu\/research\/acl-2024\/\u0022\u003E\u003Cem\u003Ehttps:\/\/sites.gatech.edu\/research\/acl-2024\/\u003C\/em\u003E\u003C\/a\u003E\u003Cem\u003E for news and coverage of Georgia Tech research presented at ACL 2024.\u003C\/em\u003E\u003C\/p\u003E","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003EA research group is calling for internet and social media moderators to strengthen their detection and intervention protocols for violent speech.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003ETheir study of language detection software found that algorithms struggle to differentiate anti-Asian violence-provoking speech from general hate speech. Left unchecked, threats of violence online can go unnoticed and turn into real-world attacks.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EResearchers from Georgia Tech and the Anti-Defamation League (ADL) teamed together\u0026nbsp;\u003Ca href=\u0022https:\/\/claws-lab.github.io\/violence-provoking-speech\/\u0022\u003E\u003Cstrong\u003Ein the study\u003C\/strong\u003E\u003C\/a\u003E. They made their discovery while testing natural language processing (NLP) models trained on data they crowdsourced from Asian communities.\u0026nbsp;\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"A study of language detection software found that algorithms struggle to differentiate anti-Asian violence-provoking speech from general hate speech. Left unchecked, threats of violence online can go unnoticed and turn into real-world attacks. "}],"uid":"36319","created_gmt":"2024-08-09 18:25:33","changed_gmt":"2024-08-12 14:35:52","author":"Bryant Wine","boilerplate_text":"","field_publication":"","field_article_url":"","dateline":{"date":"2024-08-07T00:00:00-04:00","iso_date":"2024-08-07T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"674549":{"id":"674549","type":"image","title":"stopping_asian_hate story.jpg","body":null,"created":"1723227945","gmt_created":"2024-08-09 18:25:45","changed":"1723227945","gmt_changed":"2024-08-09 18:25:45","alt":"CSE ACL 2024","file":{"fid":"258092","name":"stopping_asian_hate story.jpg","image_path":"\/sites\/default\/files\/2024\/08\/09\/stopping_asian_hate%20story.jpg","image_full_path":"http:\/\/hg.gatech.edu\/\/sites\/default\/files\/2024\/08\/09\/stopping_asian_hate%20story.jpg","mime":"image\/jpeg","size":116043,"path_740":"http:\/\/hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/2024\/08\/09\/stopping_asian_hate%20story.jpg?itok=mwTZ9fpB"}},"674550":{"id":"674550","type":"image","title":"GV ACL 2024.jpg","body":null,"created":"1723227974","gmt_created":"2024-08-09 18:26:14","changed":"1723227974","gmt_changed":"2024-08-09 18:26:14","alt":"Gaurav Verma CSE ACL 2024","file":{"fid":"258093","name":"GV ACL 2024.jpg","image_path":"\/sites\/default\/files\/2024\/08\/09\/GV%20ACL%202024.jpg","image_full_path":"http:\/\/hg.gatech.edu\/\/sites\/default\/files\/2024\/08\/09\/GV%20ACL%202024.jpg","mime":"image\/jpeg","size":99701,"path_740":"http:\/\/hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/2024\/08\/09\/GV%20ACL%202024.jpg?itok=at_-uSbG"}},"674551":{"id":"674551","type":"image","title":"SK and MDC ACL 2024.jpg","body":null,"created":"1723228196","gmt_created":"2024-08-09 18:29:56","changed":"1723228196","gmt_changed":"2024-08-09 18:29:56","alt":"Srijan Kumar CSE ACL 2024","file":{"fid":"258094","name":"SK and MDC ACL 2024.jpg","image_path":"\/sites\/default\/files\/2024\/08\/09\/SK%20and%20MDC%20ACL%202024.jpg","image_full_path":"http:\/\/hg.gatech.edu\/\/sites\/default\/files\/2024\/08\/09\/SK%20and%20MDC%20ACL%202024.jpg","mime":"image\/jpeg","size":59514,"path_740":"http:\/\/hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/2024\/08\/09\/SK%20and%20MDC%20ACL%202024.jpg?itok=OFDDw98_"}},"674552":{"id":"674552","type":"image","title":"ACL Figure.png","body":null,"created":"1723228228","gmt_created":"2024-08-09 18:30:28","changed":"1723228228","gmt_changed":"2024-08-09 18:30:28","alt":"CSE ACL 2024","file":{"fid":"258095","name":"ACL Figure.png","image_path":"\/sites\/default\/files\/2024\/08\/09\/ACL%20Figure.png","image_full_path":"http:\/\/hg.gatech.edu\/\/sites\/default\/files\/2024\/08\/09\/ACL%20Figure.png","mime":"image\/png","size":338094,"path_740":"http:\/\/hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/2024\/08\/09\/ACL%20Figure.png?itok=vNvVS4_O"}}},"media_ids":["674549","674550","674551","674552"],"related_links":[{"url":"https:\/\/www.cc.gatech.edu\/news\/study-highlights-challenges-detecting-violent-speech-aimed-asian-communities","title":"Study Highlights Challenges in Detecting Violent Speech Aimed at Asian Communities"}],"groups":[{"id":"47223","name":"College of Computing"},{"id":"1188","name":"Research Horizons"},{"id":"50877","name":"School of Computational Science and Engineering"}],"categories":[{"id":"153","name":"Computer Science\/Information Technology and Security"},{"id":"135","name":"Research"},{"id":"134","name":"Student and Faculty"},{"id":"8862","name":"Student Research"}],"keywords":[{"id":"192863","name":"go-ai"},{"id":"10199","name":"Daily Digest"},{"id":"9153","name":"Research Horizons"},{"id":"187915","name":"go-researchnews"},{"id":"166983","name":"School of Computational Science and Engineering"},{"id":"187812","name":"artificial intelligence (AI)"}],"core_research_areas":[{"id":"193655","name":"Artificial Intelligence at Georgia Tech"},{"id":"145171","name":"Cybersecurity"},{"id":"39431","name":"Data Engineering and Science"},{"id":"39501","name":"People and Technology"},{"id":"39511","name":"Public Service, Leadership, and Policy"}],"news_room_topics":[{"id":"71881","name":"Science and Technology"}],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003EBryant Wine, Communications Officer\u003Cbr\u003E\u003Ca href=\u0022mailto:bryant.wine@cc.gatech.edu\u0022\u003Ebryant.wine@cc.gatech.edu\u003C\/a\u003E\u003C\/p\u003E","format":"limited_html"}],"email":[],"slides":[],"orientation":[],"userdata":""}}}