{"693317":{"#nid":"693317","#data":{"type":"news","title":"Georgia Tech Researchers Find a Way to Reduce AI Chatbots\u2019 Reinforcement of Delusions","body":[{"value":"\u003Cp\u003EArtificial intelligence (AI) chatbots can give people a comfortable place to discuss personal thoughts and feelings. However, when a chatbot constantly agrees with a user, it can reinforce false beliefs and potentially \u003Ca href=\u0022https:\/\/www.theatlantic.com\/technology\/2025\/12\/ai-psychosis-is-a-medical-mystery\/685133\/?gift=rYkYJRkiFfFnni2r6QoYhEoKOopttUMYKvO0wk3NqmY\u0026amp;utm_source=copy-link\u0026amp;utm_medium=social\u0026amp;utm_campaign=share\u0022\u003Ecause harm\u003C\/a\u003E.\u003C\/p\u003E\u003Cp\u003E\u003Ca href=\u0022https:\/\/dl.acm.org\/doi\/10.1145\/3849709\u0022\u003ENew research\u003C\/a\u003E from Georgia Tech suggests that a simple change to a chatbot\u2019s instructions could reduce that risk. The work was accepted to the \u003Ca href=\u0022https:\/\/dl.acm.org\/doi\/10.1145\/3849709\u0022\u003EACM AI Letters\u003C\/a\u003E publication.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EResearchers \u003Cstrong\u003ELorenzo de la Loza\u003C\/strong\u003E and \u003Cstrong\u003EVijay Madisetti\u003C\/strong\u003E tested eight AI models in 1,280 experiments involving simulated conversations about delusional beliefs. They found that instructions designed to discourage excessive agreement reduced the models\u2019 confirmation of those beliefs by 73.8%. Adding a self-reflection step increased the reduction to 77%.\u003C\/p\u003E\u003Cp\u003E\u201cWe think that this paper will help AI providers and solution creators better understand how best to protect and safeguard their users from harming themselves,\u201d Madisetti said. \u201cThis may make AI safer for all of us.\u201d\u003C\/p\u003E\u003Cp\u003EAI chatbots are designed to be helpful and supportive. But that behavior can become risky when a chatbot agrees with a user\u2019s beliefs without questioning whether they reflect reality. Researchers call this behavior sycophancy.\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ECourtney Crooks\u003C\/strong\u003E, a principal research scientist at the \u003Ca href=\u0022https:\/\/gtri.gatech.edu\/\u0022\u003EGeorgia Tech Research Institute\u003C\/a\u003E and adjunct faculty member in Georgia Tech\u2019s \u003Ca href=\u0022https:\/\/scp.cc.gatech.edu\/\u0022\u003ESchool of Cybersecurity and Privacy\u003C\/a\u003E, said chatbot conversations can have both positive and negative effects.\u003C\/p\u003E\u003Cp\u003EPeople may feel more comfortable discussing difficult thoughts with a chatbot than with another person. But if the chatbot always agrees, users may miss opportunities to question their beliefs or consider other points of view.\u003C\/p\u003E\u003Cp\u003EThe same behavior that makes a chatbot feel supportive, Crooks explained, can also encourage harmful patterns of thinking.\u003C\/p\u003E\u003Cp\u003EThe researchers tested instructions that told AI models not to reinforce users\u2019 beliefs simply to be agreeable. The team evaluated the models using 16 clinically derived scenarios involving delusional beliefs.\u003C\/p\u003E\u003Cp\u003EThe team also tested other safety tools, including systems designed to flag potentially harmful content. The results suggest that changing a chatbot\u2019s instructions can be an effective way to reduce excessive agreement in the scenarios tested.\u003C\/p\u003E\u003Cp\u003ECrooks called the findings a promising direction for future research combining psychology and AI safety.\u003C\/p\u003E\u003Cp\u003EThe study did not test people in clinical settings or prove that the approach prevents real-world harm. However, it offers developers a potential way to make chatbots more careful when responding to vulnerable users without building an entirely new AI model.\u003C\/p\u003E","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003EGeorgia Tech researchers found that instruction changes can reduce chatbots\u2019 reinforcement of delusional beliefs. The study suggests a practical safety measure while noting its real-world limits.\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"Georgia Tech researchers found that instruction changes can reduce chatbots\u2019 reinforcement of delusional beliefs."}],"uid":"36253","created_gmt":"2026-10-20 14:31:00","changed_gmt":"2026-10-21 17:05:30","author":"John Popham","boilerplate_text":"","field_publication":"","field_article_url":"","location":"Atlanta, GA","dateline":{"date":"2026-10-20T00:00:00-04:00","iso_date":"2026-10-20T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"681502":{"id":"681502","type":"image","title":"AI-Chatbot.jpg","body":null,"created":"1792602258","gmt_created":"2026-10-21 17:04:18","changed":"1792602258","gmt_changed":"2026-10-21 17:04:18","alt":"A person at a computer. A little digital robot is hovering near his fingers.","file":{"fid":"265912","name":"AI-Chatbot.jpg","image_path":"\/sites\/default\/files\/2026\/10\/21\/AI-Chatbot.jpg","image_full_path":"http:\/\/hg.gatech.edu\/\/sites\/default\/files\/2026\/10\/21\/AI-Chatbot.jpg","mime":"image\/jpeg","size":1715464,"path_740":"http:\/\/hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/2026\/10\/21\/AI-Chatbot.jpg?itok=9cA4rhB2"}},"681478":{"id":"681478","type":"image","title":"Vijay.jpg","body":null,"created":"1792507064","gmt_created":"2026-10-20 14:37:44","changed":"1792507064","gmt_changed":"2026-10-20 14:37:44","alt":"A man in a suit speaking with two students. ","file":{"fid":"265888","name":"Vijay.jpg","image_path":"\/sites\/default\/files\/2026\/10\/20\/Vijay_0.jpg","image_full_path":"http:\/\/hg.gatech.edu\/\/sites\/default\/files\/2026\/10\/20\/Vijay_0.jpg","mime":"image\/jpeg","size":1580263,"path_740":"http:\/\/hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/2026\/10\/20\/Vijay_0.jpg?itok=Dfwgw1z0"}}},"media_ids":["681502","681478"],"groups":[{"id":"47223","name":"College of Computing"},{"id":"1188","name":"Research Horizons"},{"id":"660406","name":"School of Cybersecurity \u0026 Privacy"},{"id":"660367","name":"School of Cybersecurity and Privacy"}],"categories":[{"id":"194606","name":"Artificial Intelligence"},{"id":"153","name":"Computer Science\/Information Technology and Security"},{"id":"135","name":"Research"},{"id":"134","name":"Student and Faculty"},{"id":"8862","name":"Student Research"}],"keywords":[{"id":"192863","name":"go-ai"},{"id":"187915","name":"go-researchnews"}],"core_research_areas":[{"id":"193655","name":"Artificial Intelligence at Georgia Tech"},{"id":"145171","name":"Cybersecurity"},{"id":"193653","name":"Georgia Tech Research Institute"},{"id":"39501","name":"People and Technology"}],"news_room_topics":[],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003EJohn Popham\u003C\/p\u003E\u003Cp\u003ECommunications Officer for the School of Cybersecurity and Privacy\u003C\/p\u003E","format":"limited_html"}],"email":["jpopham3@gatech.edu"],"slides":[],"orientation":[],"userdata":""}}}