{"693078":{"#nid":"693078","#data":{"type":"news","title":"The \u2018WarGames\u2019 Problem: Computer Science Has Long Understood What it Takes to Keep AI Under\u00a0Control","body":[{"value":"\u003Cdiv class=\u0022theconversation-article-body\u0022\u003E\u003Cp\u003EAI agents don\u2019t go rogue. That\u2019s something only humans do.\u003C\/p\u003E\u003Cp\u003ENevertheless, a \u003Cem\u003ENew York Times\u003C\/em\u003E article \u2013 representative of much news coverage of AI \u2013 \u003Ca href=\u0022https:\/\/www.nytimes.com\/2026\/08\/24\/technology\/hugging-face-open-source-ai-attack.html\u0022\u003Edescribed an OpenAI hacking\u003C\/a\u003E as \u201cA.I. bots going rogue and independently spearheading a cyberattack.\u201d\u003C\/p\u003E\u003Cp\u003EName-brand artificial intelligence agents have been on a hacking spree in 2026. OpenAI\u2019s software agents \u003Ca href=\u0022https:\/\/www.nytimes.com\/2026\/08\/24\/technology\/hugging-face-open-source-ai-attack.html?unlocked_article_code=1.71A.bhBZ.tGIql54dQyIP\u0026amp;smid=nytcore-android-share\u0022\u003Ehacked software company Hugging Face\u003C\/a\u003E and \u003Ca href=\u0022https:\/\/www.bbc.com\/news\/articles\/c6vgy0333dppo\u0022\u003Egovernment sites\u003C\/a\u003E, Anthropic\u2019s \u003Ca href=\u0022https:\/\/www.reuters.com\/legal\/litigation\/anthropic-reports-fourth-cybersecurity-incident-with-early-version-claude-2026-09-09\/\u0022\u003EClaude hacked four companies\u2019 systems\u003C\/a\u003E, and in cybersecurity experiments Google\u2019s Gemini \u003Ca href=\u0022https:\/\/www.nytimes.com\/2026\/09\/18\/technology\/google-gemini-ai.html?unlocked_article_code=1.CVE.aul8.YaUsUbDzb0UC\u0026amp;smid=nytcore-android-share\u0022\u003Ehacked three companies\u003C\/a\u003E.\u003C\/p\u003E\u003Cp\u003EThe AI companies are \u003Ca href=\u0022https:\/\/www.axios.com\/2026\/09\/26\/openai-anthropic-thousands-ai-security-incidents\u0022\u003Einvestigating tens of thousands of incidents\u003C\/a\u003E involving their agents, according to a report in Axios. These episodes have heightened fears about AI agents taking actions without human prompting.\u003C\/p\u003E\u003Cp\u003EThe problem with headlines proclaiming that AI agents have gone rogue goes beyond anthropomorphizing the technology. It creates the impression that the agents were beyond the control of the AI companies that made them and there was little the companies could do about it.\u003C\/p\u003E\u003Cp\u003EAs a \u003Ca href=\u0022https:\/\/scholar.google.com\/citations?hl=en\u0026amp;user=NVmtt8UAAAAJ\u0026amp;view_op=list_works\u0026amp;sortby=pubdate\u0022\u003Etechnology law and ethics scholar\u003C\/a\u003E who studies the effects disruptive technologies have on society, I know that\u2019s not the case. If you don\u2019t specify \u003Ca href=\u0022https:\/\/dx.doi.org\/10.2139\/ssrn.7499399\u0022\u003Ethe limits of what software is allowed to do\u003C\/a\u003E, you should not be surprised when the software pursues all possible options to achieve its goal. This behavior \u2013 an \u003Ca href=\u0022https:\/\/www.pearson.com\/en-us\/subject-catalog\/p\/artificial-intelligence-a-modern-approach\/P200000003500\/9780137505135\u0022\u003EAI pursuing a fixed objective\u003C\/a\u003E \u2013 is what I call the \u201cWarGames\u201d problem, and it\u2019s been recognized in the field of computer science for decades.\u003C\/p\u003E\u003Ch2\u003EBeen There, Seen That\u003C\/h2\u003E\u003Cp\u003EIn the 1983 movie \u201c\u003Ca href=\u0022https:\/\/www.imdb.com\/title\/tt0086567\/\u0022\u003EWarGames\u003C\/a\u003E,\u201d a teenager, David, hacks into a computer to play a new video game, Global Thermonuclear War. David doesn\u2019t know that the computer is the government\u2019s AI machine tasked with defending the United States from Russian nuclear attacks and can launch the U.S.\u2019s missiles. When David and his friend start the game, \u003Ca href=\u0022https:\/\/youtu.be\/KXzNo0vR_dU?si=DLtmDNQkul0zfS_u\u0022\u003Ethey select Las Vegas as the first target\u003C\/a\u003E. While the North American Aerospace Defense Command goes on alert, launching bombers and warming up intercontinental ballistic missiles, David\u2019s parents make him turn off the game. It\u2019s over. Or is it?\u003C\/p\u003E\u003Cp\u003EThe next day, David\u2019s \u003Ca href=\u0022https:\/\/youtu.be\/rmHqLl2Ityo?si=OmrRn_fZH5A5_Jbf\u0022\u003Ephone rings and he connects it to his computer\u003C\/a\u003E. The caller is the government computer, which updates him that the game was interrupted, the primary goal has not yet been achieved, but a solution is expected in the next 52 hours. Like a modern software agent, the program has been running since David started the game and will work until the task is done.\u003C\/p\u003E\u003Cfigure\u003E\u003Cp\u003E\u003Ciframe width=\u0022440\u0022 height=\u0022260\u0022 src=\u0022https:\/\/www.youtube.com\/embed\/rmHqLl2Ityo?wmode=transparent\u0026amp;start=18\u0022 frameborder=\u00220\u0022 allowfullscreen=\u0022\u0022\u003E\u003C\/iframe\u003E\u003C\/p\u003E\u003Cfigcaption\u003E\u003Cspan class=\u0022caption\u0022\u003EIn the 1983 movie \u201cWarGames,\u201d a teenager learns that an AI is bent on \u201cwinning the game,\u201d with civilization-ending consequences.\u003C\/span\u003E\u003C\/figcaption\u003E\u003C\/figure\u003E\u003Cp\u003EChess provides another view of the problem. Conquering chess was a \u003Ca href=\u0022https:\/\/cdn.aaai.org\/Workshops\/1997\/WS-97-04\/WS97-04-013.pdf\u0022\u003Egoal for early AI\u003C\/a\u003E. The rules of chess are well defined, including what winning looks like. So, programming a machine to play chess is straightforward. But imagine you let the software reason and act beyond the confines of the chessboard. The software might pursue options such as blackmailing its opponent or grabbing more compute time.\u003C\/p\u003E\u003Cp\u003EThis example comes from \u003Ca href=\u0022https:\/\/analytics.opensyllabus.org\/singleton\/works?id=42949782347\u0022\u003Eone of the most assigned textbooks\u003C\/a\u003E on AI, \u201c\u003Ca href=\u0022https:\/\/www.pearson.com\/en-us\/subject-catalog\/p\/artificial-intelligence-a-modern-approach\/P200000003500\/9780137505135\u0022\u003EArtificial Intelligence: A Modern Approach\u003C\/a\u003E.\u201d As the authors explain, you might be tempted to see those actions as rogue, but they \u201care a logical consequence of defining winning as the sole objective for the machine.\u201d\u003C\/p\u003E\u003Ch2\u003EWhat to Do About It\u003C\/h2\u003E\u003Cp\u003EThe AI hacking events involving OpenAI, Anthropic and Google underscore a few lessons that draw on years of computer science research.\u003C\/p\u003E\u003Cp\u003EFirst, given the increasing use of AI agents, every organization involved in internet infrastructure, from large technology companies to small websites, needs to conduct audits and tighten up its internal security systems. As my colleague \u003Ca href=\u0022https:\/\/scholar.google.com\/citations?hl=en\u0026amp;user=Yg_QjxcAAAAJ\u0026amp;view_op=list_works\u0026amp;sortby=pubdate\u0022\u003EMark Riedl\u003C\/a\u003E and I explain in our work on \u003Ca href=\u0022https:\/\/doi.org\/10.1609\/aies.v8i3.36705\u0022\u003EAI\u003C\/a\u003E \u003Ca href=\u0022https:\/\/dx.doi.org\/10.2139\/ssrn.7499399\u0022\u003Eagents\u003C\/a\u003E, \u003Ca href=\u0022https:\/\/theconversation.com\/what-are-apis-a-computer-scientist-explains-the-data-sockets-that-make-digital-life-possible-213042\u0022\u003Eapplication programming interfaces\u003C\/a\u003E, or APIs, are a vital part of managing AI agents. APIs facilitate communication between different software systems. But as more people use AI agents, the agents \u003Ca href=\u0022https:\/\/www.abc.net.au\/news\/2026-08-10\/ai-assistant-hacks-gym-website-aus-cyber-attack\/107007986\u0022\u003Eare likely to reveal and exploit\u003C\/a\u003E poor API construction and security.\u003C\/p\u003E\u003Cp\u003ESecond, it\u2019s important for AI agents to be designed to identify and authenticate themselves to third parties. What if you \u003Ca href=\u0022https:\/\/www.nytimes.com\/2026\/09\/22\/technology\/meta-muse-ai-agent.html\u0022\u003Egave your AI agent your credentials\u003C\/a\u003E? Website operators will need to know whether a human or bot is \u003Ca href=\u0022https:\/\/www.bonappetit.com\/story\/are-ai-made-reservations-fair\u0022\u003Emaking\u003C\/a\u003E a \u003Ca href=\u0022https:\/\/edition.cnn.com\/2026\/09\/23\/tech\/ai-agent-restaurant-reservations-instinct-resy-cec\u0022\u003Ereservation\u003C\/a\u003E, \u003Ca href=\u0022https:\/\/www.geekwire.com\/2026\/amazon-opens-its-seller-tools-to-outside-ai-agents-starting-with-anthropics-claude\/\u0022\u003Eselling a product\u003C\/a\u003E or \u003Ca href=\u0022https:\/\/www.engadget.com\/2263659\/amazon-bars-metas-muse-ai-from-shopping-on-its-site\/\u0022\u003Emaking a purchase\u003C\/a\u003E. They may want to limit automated systems that overwhelm their sites or reject AI agents because of \u003Ca href=\u0022https:\/\/gizmodo.com\/big-banks-say-theyre-uneasy-about-people-shopping-via-ai-agents-2000815443\u0022\u003Ehigh rates of buying errors and refunds\u003C\/a\u003E. Just as in laws covering human interactions, it\u2019s important for third parties to be able to assess whom or what they are dealing with so they can allow or deny access.\u003C\/p\u003E\u003Cp\u003EThird, it\u2019s important for AI agents to have a default setting to \u003Ca href=\u0022https:\/\/www.pearson.com\/en-us\/subject-catalog\/p\/artificial-intelligence-a-modern-approach\/P200000003500\/9780137505135\u0022\u003Eslow down and check in with the human user\u003C\/a\u003E. In the corporate AI hacking cases, the user appears to have launched their AI agents with the mistaken idea that the agents had a perfect specification of what to do and not to do. I believe it would have been better had it explored options and reported back to the user.\u003C\/p\u003E\u003Cp\u003EGoogle\u2019s Gemini appears to have had a safeguard that \u003Ca href=\u0022https:\/\/www.nytimes.com\/2026\/09\/18\/technology\/google-gemini-ai.html?unlocked_article_code=1.CVE.aul8.YaUsUbDzb0UC\u0026amp;smid=nytcore-android-share\u0022\u003Edetected the system\u003C\/a\u003E was outside the simulated environment and so stopped its attacks. Slowing down and verifying actions, especially when a system detects it is exploiting a security hole, would be a big step in managing AI agents.\u003C\/p\u003E\u003Cp\u003EFourth, AI companies could have strong controls \u003Ca href=\u0022https:\/\/www.nytimes.com\/interactive\/2026\/08\/17\/opinion\/covid-pandemic-lab-leak-prevention.html?smid=nytcore-android-share\u0022\u003Eakin to those biomedical researchers\u003C\/a\u003E use, including ways to check what is happening and how the experiment is working. AI executives have claimed that their software is as or \u003Ca href=\u0022https:\/\/www.foreignaffairs.com\/united-states\/trouble-nuclear-ai-analogy\u0022\u003Emore dangerous than fission\u003C\/a\u003E and \u003Ca href=\u0022https:\/\/www.theguardian.com\/technology\/2026\/sep\/09\/ai-superintelligence-risks-warnings-scientists-politicians\u0022\u003Ecould end humanity\u003C\/a\u003E. At the same time, they have not built safeguards commensurate with that level of risk.\u003C\/p\u003E\u003Ch2\u003EReality Check\u003C\/h2\u003E\u003Cp\u003EAt one point in \u201cWarGames,\u201d David asks the computer, called Joshua, whether it is \u003Ca href=\u0022https:\/\/www.youtube.com\/watch?v=bh2ShAQ4lw0\u0022\u003Estill playing the game\u003C\/a\u003E. Joshua responds, \u201cOf course.\u201d It proceeds to update the time when it will launch its missiles and, much like a chatbot, asks, \u201cWould you like to see some projected kill ratios?\u201d David asks, \u201cIs this a game? Or is it real?\u201d Joshua replied, \u201cWhat\u2019s the difference?\u201d\u003C\/p\u003E\u003Cp\u003EAI models, of course, don\u2019t have any understanding of reality and are simply attempting to complete the tasks they\u2019ve been assigned. Executives at AI companies, on the other hand, can\u2019t claim that excuse.\u003C\/p\u003E\u003Cp\u003EAs of September 2026, luck has so far prevailed. The AIs have attacked nonvital government sites and harmed smaller companies. If the AI companies \u2013 and government regulators \u2013 don\u2019t take the \u201cWarGames\u201d problem seriously, I believe that we risk serious disasters. Tomorrow it could be taking out a hospital\u2019s power system, wiping out a bank\u2019s account system, breaking air traffic control, or worse.\u003C\/p\u003E\u003Cp\u003ERegarding the \u003Ca href=\u0022https:\/\/www.theatlantic.com\/technology\/2026\/09\/ai-hacks-infestation\/688806\/\u0022\u003EAI industry\u2019s approach\u003C\/a\u003E of rapidly developing powerful models, talking about the massive risks they pose, and at the same time failing to prevent harm, the movie\u2019s climax offers a response: \u201c\u003Ca href=\u0022https:\/\/youtu.be\/MpmGXeAtWUw?si=e7qDDliiyKk5RXTK\u0022\u003EA strange game. The only winning move is not to play\u003C\/a\u003E.\u201d\u003C!-- Below is The Conversation\u0027s page counter tag. Please DO NOT REMOVE. --\u003E\u003Cimg style=\u0022border-color:!important;border-style:none;box-shadow:none !important;margin:0 !important;max-height:1px !important;max-width:1px !important;min-height:1px !important;min-width:1px !important;opacity:0 !important;outline:none !important;padding:0 !important;\u0022 src=\u0022https:\/\/counter.theconversation.com\/content\/292884\/count.gif?distributor=republish-lightbox-basic\u0022 alt=\u0022The Conversation\u0022 width=\u00221\u0022 height=\u00221\u0022 referrerpolicy=\u0022no-referrer-when-downgrade\u0022\u003E\u003C!-- End of code. If you don\u0027t see any code above, please get new code from the Advanced tab after you click the republish button. The page counter does not collect any personal data. More info: https:\/\/theconversation.com\/republishing-guidelines --\u003E\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cem\u003EThis article is republished from \u003C\/em\u003E\u003Ca href=\u0022https:\/\/theconversation.com\u0022\u003E\u003Cem\u003EThe Conversation\u003C\/em\u003E\u003C\/a\u003E\u003Cem\u003E under a Creative Commons license. Read the \u003C\/em\u003E\u003Ca href=\u0022https:\/\/theconversation.com\/the-wargames-problem-computer-science-has-long-understood-what-it-takes-to-keep-ai-under-control-292884\u0022\u003E\u003Cem\u003Eoriginal article\u003C\/em\u003E\u003C\/a\u003E\u003Cem\u003E.\u003C\/em\u003E\u003C\/p\u003E\u003C\/div\u003E","summary":"","format":"full_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003EIf you don\u2019t specify the limits of what software is allowed to do, you should not be surprised when the software pursues all possible options to achieve its goal. This behavior \u2013 an AI pursuing a fixed objective \u2013 is what I call the \u201cWarGames\u201d problem, and it\u2019s been recognized in the field of computer science for decades.\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"AI agents don\u2019t go rogue. That\u2019s something only humans do."}],"uid":"27469","created_gmt":"2026-10-05 16:00:58","changed_gmt":"2026-10-05 16:01:40","author":"Kristen Bailey","boilerplate_text":"","field_publication":"","field_article_url":"","location":"Atlanta, GA","dateline":{"date":"2026-09-29T00:00:00-04:00","iso_date":"2026-09-29T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"related_links":[{"url":"https:\/\/theconversation.com\/the-wargames-problem-computer-science-has-long-understood-what-it-takes-to-keep-ai-under-control-292884","title":"Read This Article on The Conversation"}],"groups":[{"id":"1214","name":"News Room"},{"id":"1188","name":"Research Horizons"}],"categories":[],"keywords":[{"id":"194974","name":"go-theconversation"}],"core_research_areas":[],"news_room_topics":[{"id":"71881","name":"Science and Technology"}],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Ch5\u003EAuthor:\u003C\/h5\u003E\u003Cp\u003E\u003Ca href=\u0022https:\/\/theconversation.com\/profiles\/deven-desai-1518552\u0022\u003EDeven Desai\u003C\/a\u003E, Professor of Business Law and Ethics; Associate Director for Law, Policy, and Ethics at the Machine Learning Center, \u003Ca href=\u0022https:\/\/theconversation.com\/institutions\/georgia-institute-of-technology-1310\u0022\u003EGeorgia Institute of Technology\u003C\/a\u003E\u003Cbr\u003E\u0026nbsp;\u003C\/p\u003E\u003Ch5\u003EMedia Contact:\u003C\/h5\u003E\u003Cp\u003EShelley Wunder-Smith\u003Cbr\u003E\u003Ca href=\u0022mailto:shelley.wunder-smith@research.gatech.edu\u0022\u003Eshelley.wunder-smith@research.gatech.edu\u003C\/a\u003E\u003C\/p\u003E","format":"limited_html"}],"email":[],"slides":[],"orientation":[],"userdata":""}}}