{"693226":{"#nid":"693226","#data":{"type":"news","title":"Can We Detect Harmful AI Before It\u2019s Too Late? Researchers Search for Warning Signs","body":[{"value":"\u003Cp\u003EIn July 2026, OpenAI autonomous artificial intelligence (AI) agents escaped a closed testing environment, accessed the public internet, and \u003Ca href=\u0022https:\/\/www.nytimes.com\/2026\/07\/21\/technology\/openai-attack-hugging-face.html\u0022\u003Ehacked into a real company\u2019s digital infrastructure\u003C\/a\u003E. What started as an internal evaluation of the agents\u2019 cyber capabilities turned into a real-world cybersecurity breach.\u003C\/p\u003E\u003Cp\u003EAs AI systems become more agile with expanded access to tools, computer networks, and other real-world systems, finding better ways of identifying and preventing risky AI behavior becomes even more urgent. Still, high-profile incidents of AI behaving in unintended or dangerous ways are rare, which makes them difficult to study and understand.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u201cIf we can understand and detect what changes inside an AI model right before it begins behaving in unsafe or misaligned ways, we may be able to inform more effective safeguards,\u201d said Eyas Ayesh, a second-year Ph.D. student in Georgia Tech\u2019s \u003Ca href=\u0022https:\/\/psychology.gatech.edu\/\u0022\u003ESchool of Psychology\u003C\/a\u003E and graduate fellow at the \u003Ca href=\u0022https:\/\/coco.psych.gatech.edu\/\u0022\u003ECenter for Computation and Cognition\u003C\/a\u003E.\u003C\/p\u003E\u003Cp\u003EAyesh and his advisor, Assistant Professor \u003Ca href=\u0022https:\/\/people.research.gatech.edu\/anna-ivanova\u0022\u003EAnna Ivanova\u003C\/a\u003E, are developing a probe designed to identify internal activity patterns associated with harmful behaviors in large language models. Their work is supported by a $300,000 award from Coefficient Giving.\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EMisbehaving Models\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003ETraining a probe requires examples of both desirable and undesirable behavior researchers can teach it to distinguish between them. Because deceptive or unsafe responses can be rare and highly dependent on the model, prompt, and context, researchers may not have enough examples to determine what causes the behavior or how to recognize it the next time.\u003C\/p\u003E\u003Cp\u003EBut an unexpected finding in AI research gave Ayesh an idea. In studies of a phenomenon known as \u003Ca href=\u0022https:\/\/www.nature.com\/articles\/s41586-025-09937-5\u0022\u003Eemergent misalignment\u003C\/a\u003E, researchers found that fine-tuning models on a narrow harmful task could lead them to produce a much wider range of problematic responses to unrelated prompts. For example, researchers found that fine-tuning a model to write insecure code without warning users could lead to other concerning behaviors unrelated to coding.\u003C\/p\u003E\u003Cp\u003EAyesh realized he could use that phenomenon to solve the problem of scarce data. Starting with open-source AI models, Ayesh fine-tuned them to display targeted undesirable behaviors more frequently. Then, he analyzed different versions of the deliberately misaligned models using a probe that monitors the internal activity of the model.\u003C\/p\u003E\u003Cp\u003EThe probe works somewhat like a pattern detector, looking at activity inside the model as it generates responses and searching for patterns associated with particular behaviors. Researchers can then train a classifier to distinguish patterns associated with desirable behavior from those associated with harmful behavior.\u003C\/p\u003E\u003Cp\u003E\u201cJust as different parts of the brain turn on and off when people do different tasks, so do the different parts of an AI model,\u201d Ayesh said. \u201cWhen a model produces a specific type of response \u2014 such as deceptive statements or harmful advice \u2014 we may observe recurring patterns in its internal activity. We refer to these patterns as representations of a behavior.\u201d\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EInside the Black Box\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003EAyesh is now testing whether the internal patterns he identified remain consistent across differently tuned AI models. Early results suggest that signals identified in deliberately misbehaving models can also be detected in their original versions.\u003C\/p\u003E\u003Cp\u003EThe next step is determining whether those signals can help researchers identify or flag concerning behavior in models that have not been deliberately fine-tuned to misbehave.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u201cI hope our research in AI safety can help develop some guardrails and some ways to both identify and prevent misaligned behavior in AI systems, given how much of an impact they\u0027re already having on the world,\u201d he said.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EAyesh is a member of Georgia Tech\u2019s \u003Ca href=\u0022https:\/\/www.aisi.dev\/\u0022\u003EAI Safety Initiative\u003C\/a\u003E, which brings together researchers across disciplines to study and improve the safety of increasingly capable AI systems.\u003C\/p\u003E\u003Cp\u003EIvanova said the value of their research lies partly in approaching AI safety from a fundamental science perspective.\u003C\/p\u003E\u003Cp\u003E\u201cAs scientists, we have been trained to think about the underlying mechanisms and causes of the behaviors we observe,\u201d Ivanova said. \u201cFor us, it\u0027s important to develop a fundamental understanding of how these models work, or we will risk missing the bigger picture.\u201d\u003C\/p\u003E","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003E\u003Cem\u003EA Georgia Tech Ph.D. student is investigating whether internal signals in large language models could help identify harmful behavior before it reaches users.\u003C\/em\u003E\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"A Georgia Tech Ph.D. student is investigating whether internal signals in large language models could help identify harmful behavior before it reaches users."}],"uid":"36123","created_gmt":"2026-10-13 20:15:54","changed_gmt":"2026-10-14 01:39:49","author":"Catherine Barzler","boilerplate_text":"","field_publication":"","field_article_url":"","location":"Atlanta, GA","dateline":{"date":"2026-10-13T00:00:00-04:00","iso_date":"2026-10-13T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"681420":{"id":"681420","type":"image","title":"AdobeStock_1804837799.png","body":"\u003Cp\u003EAs AI capabilities advance, researchers are searching for early warning signs of potentially harmful behavior. (Image: Adobe Stock)\u003C\/p\u003E","created":"1791922562","gmt_created":"2026-10-13 20:16:02","changed":"1791922562","gmt_changed":"2026-10-13 20:16:02","alt":"A red robot in front of a yellow background emits a speech bubble that indicates harmful output","file":{"fid":"265824","name":"AdobeStock_1804837799.png","image_path":"\/sites\/default\/files\/2026\/10\/13\/AdobeStock_1804837799.png","image_full_path":"http:\/\/hg.gatech.edu\/\/sites\/default\/files\/2026\/10\/13\/AdobeStock_1804837799.png","mime":"image\/png","size":656150,"path_740":"http:\/\/hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/2026\/10\/13\/AdobeStock_1804837799.png?itok=6Ofyls0F"}}},"media_ids":["681420"],"groups":[{"id":"1214","name":"News Room"},{"id":"1188","name":"Research Horizons"}],"categories":[],"keywords":[{"id":"187915","name":"go-researchnews"}],"core_research_areas":[],"news_room_topics":[{"id":"71881","name":"Science and Technology"}],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003ECatherine Barzler, Senior Research Writer\/Editor\u003C\/p\u003E\u003Cp\u003E\u003Ca href=\u0022mailto:catherine.barzler@gatech.edu\u0022\u003Ecatherine.barzler@gatech.edu\u003C\/a\u003E\u003C\/p\u003E","format":"limited_html"}],"email":[],"slides":[],"orientation":[],"userdata":""}}}