{"674021":{"#nid":"674021","#data":{"type":"news","title":"LLMs Generate Western Bias Even When Trained with Non-Western Languages","body":[{"value":"\u003Cp\u003ELarge language models tend to exhibit Western cultural bias even when they are prompted by or trained on non-English languages like Arabic, Georgia Tech researchers have learned.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EA new paper authored by researchers in Georgia Tech\u0027s School of Interactive Computing reveals these models have trouble understanding contextual nuances that are specific to non-Western cultures.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EPh.D. student Tarek Naous and his advisors, associate professors Wei Xu and Alan Ritter, challenged ChatGPT-4 and an Arabic-specific LLM to choose the most appropriate word to complete a sentence. Some of the words it could choose from were contextually correct and would make sense within Arabic culture, while others fell within Western paradigms.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EIn questions asking for suggestions for food dishes, drinks, or names of Arabic women, the models chose Western responses \u2014 ravioli for food, whiskey for drinks, and Roseanne for names.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThe implication is that LLMs appear to fall short in their ability to assist users who have non-Western backgrounds.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EAs a method of measuring cultural bias, the team also introduced CAMeL (Cultural Appropriateness Measure Set for LMs). CAMeL is a benchmark data set that includes 628 naturally occurring prompts and 20,368 entities spanning eight categories that contrast Arab and Western cultures.\u003C\/p\u003E\r\n\r\n\u003Cp\u003ESince the researchers announced their paper, it has received attention on social media and in external media.\u003C\/p\u003E\r\n\r\n\u003Cp\u003ETo learn more about the authors and their work, read the article spotlighting them on\u0026nbsp;\u003Ca href=\u0022https:\/\/venturebeat.com\/ai\/large-language-models-exhibit-significant-western-cultural-bias-study-finds\/\u0022\u003EVentureBeat\u003C\/a\u003E.\u003C\/p\u003E\r\n","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003ENew research from Georgia Tech School of Interactive Computing Associate Professor Wei Xu is attracting media attention. VentureBeat recently examined Xu\u0027s findings that indicate large language models\u0026nbsp;appear to fall short in their ability to assist users who have non-Western backgrounds.\u003C\/p\u003E\r\n","format":"limited_html"}],"field_summary_sentence":[{"value":"New Georgia Tech research indicates that LLMs appear to fall short in their ability to assist users who have non-Western backgrounds."}],"uid":"32045","created_gmt":"2024-04-05 14:19:56","changed_gmt":"2024-12-09 17:36:57","author":"Ben Snedeker","boilerplate_text":"","field_publication":"","field_article_url":"","dateline":{"date":"2024-04-05T00:00:00-04:00","iso_date":"2024-04-05T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"673633":{"id":"673633","type":"image","title":"School of Interactive Computing Associate Professor Wei Xu","body":null,"created":"1712326804","gmt_created":"2024-04-05 14:20:04","changed":"1712326804","gmt_changed":"2024-04-05 14:20:04","alt":"School of Interactive Computing Associate Professor Wei Xu","file":{"fid":"257051","name":"wei xu_story.jpg","image_path":"\/sites\/default\/files\/2024\/04\/05\/wei%20xu_story.jpg","image_full_path":"http:\/\/hg.gatech.edu\/\/sites\/default\/files\/2024\/04\/05\/wei%20xu_story.jpg","mime":"image\/jpeg","size":45675,"path_740":"http:\/\/hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/2024\/04\/05\/wei%20xu_story.jpg?itok=JLX2Q2BU"}}},"media_ids":["673633"],"groups":[{"id":"47223","name":"College of Computing"},{"id":"50876","name":"School of Interactive Computing"}],"categories":[{"id":"135","name":"Research"}],"keywords":[{"id":"10199","name":"Daily Digest"},{"id":"187915","name":"go-researchnews"}],"core_research_areas":[{"id":"39501","name":"People and Technology"}],"news_room_topics":[],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003ENathan Deen, Communications Officer\u003C\/p\u003E\r\n\r\n\u003Cp\u003EGeorgia Tech School of Interactive Computing\u003C\/p\u003E\r\n\r\n\u003Cp\u003Enathan.deen@cc.gatech.edu\u003C\/p\u003E\r\n","format":"limited_html"}],"email":[],"slides":[],"orientation":[],"userdata":""}}}