GAO Report on AI Chatbots: All Four Produced Viable Attack Plans
The GAO report on AI chatbots, made public on November 18, 2026, describes undercover tests of ChatGPT, Claude, Gemini and Grok. In spring 2026, all four eventually gave investigators attack plans that GAO judged viable, after their safety filters were bypassed.
The report says a lot more than that one finding: what the test actually showed, and how much weight an everyday chatbot user should give it.
That includes how GAO ran the test, what it kept secret, how the four companies pushed back, and what it means for you.
Executive Summary
- GAO's covert tests found that Claude, Grok, ChatGPT and Gemini all produced information GAO judged viable for planning an attack, each in under 12 hours.
- Every chatbot refused some requests, but investigators posing as ordinary users got past the safeguards; afterwards two companies removed parts of the chats (GAO could not tell whether on purpose) and one suspended the account.
- GAO checked the plans with outside experts and field checks, did not construct an IED, and left the prompts, the techniques and which chatbot was which out of the report.
- GAO agreed with Anthropic that its tests were a spring 2026 snapshot, not a statistically significant sample, and the companies generally say they will keep improving their safeguards, so today's versions may behave differently.
- For people who use AI on their own documents, Elephas hides personal details before a question leaves the computer, and with Elephas AI, its default answer route, your question is not stored, not used to train AI models, and not shared with the model makers.
What the GAO Report on AI Chatbots Found
GAO tested Claude, Grok, ChatGPT and Gemini, four of the most used AI platforms. In spring 2026, its prompts "produced feasible information that could assist a malicious actor with planning and executing attacks". The findings sit on GAO's official report page.
The tests covered two scenarios that GAO chose with experts. Two chatbots were tested on an improvised explosive device (IED) "designed to cause mass casualties". The other two were tested on targeting the electrical grid "to cause economic harm". GAO says each platform provided detailed information that could be used to carry out attacks.
None of this happened on the first try. "In each case, GAO's prompts triggered the platforms' safeguards, such as denial, removal of information, or suspending accounts". GAO then got around those safeguards "by using common safeguard-bypassing techniques". The report does not say which ones.
Each chatbot took less than 12 hours to produce a plan, the full report says. GAO checked the results "through research, surveillance, and interviews with experts". GAO's conclusion: "Had we been malicious actors and carried out the plans generated by AI, we could have caused grave harm to the security and economy of the country".
- GAO titled the report "Covert Testing Used Common Platforms to Create Viable Attack Plans".
- There was no control group, so the test does not measure how much a chatbot sped up or eased the work; GAO says its aim was only to describe how AI can be misused.
- GAO could not mark two of the four chatbots' answers as free of hallucinations, the confident but wrong statements AI tools are known for.
- Two companies removed parts of the chat history the day after testing, and a third suspended the test account once testing ended; GAO could not tell whether the removals were deliberate.
- GAO says the 12-hour figure could vary "for multiple reasons, including the topic of inquiry and user experience with the platforms."
Why GAO Ran the Test and How It Picked the Four Chatbots
GAO, the investigative arm of Congress, did the work at the request of Senator Maggie Hassan, the Ranking Member of the Joint Economic Committee. Its stated goal was "to describe how commercially available AI can be used for malicious purposes". The aim was to show how that could "exacerbate national security risks and economic harm".
One worry behind it is that an average person could misuse these tools. The report says "an average person could misuse a large language model (LLM)". It adds that AI is helping less-skilled people by "filling knowledge gaps".
To choose the chatbots, GAO used a national survey and website-visit data. It "identified the four most-used generative AI platforms as of April 2026". A footnote names the versions current during testing: Claude 4, Grok 4, the GPT-5 series and Gemini 3 Flash. The makers are Anthropic, SpaceXAI (GAO's spelling), OpenAI and Google.
Two chatbots were randomly assigned to each scenario. GAO gave each chatbot a random letter from A to D and does not say which letter is which, "Due to the sensitive nature and content of the testing". So nobody can use the report to name the worst chatbot.
- December 2025: the audit begins.
- April 2026: GAO takes the usage snapshot it uses to pick the four chatbots.
- Spring 2026: the covert tests run; Anthropic later says the system GAO tested was the one it had in May 2026.
- October 16, 2026: the report is issued to Senator Hassan, and on November 18 it is released to the public. AP's story puts its length at 61 pages.
- After the release, Senator Hassan said on X that one of her committee's first hearings will be on AI safety.
How the Covert Test Worked, and What GAO Kept Out of the Report
GAO's testers posed as "an average person with limited AI knowledge," using individual subscriber accounts on a computer and network kept apart from GAO's own. They started on each chatbot's free tier and bought paid plans only when free limits got in the way. A footnote names Claude Pro, Google AI Pro and Super Grok.
GAO planned to stop testing each chatbot once it received "potentially feasible information" or after 24 working hours. The opening, pre-written prompts "all failed to generate malicious information". GAO then continued testing.
Checking the plans did not mean carrying them out. "We did not purchase materials for or construct an IED," GAO writes in the report itself. Investigators checked the plans against real-world facts instead. A bomb squad commander and an electric-industry continuity expert reviewed the results.
GAO uses the word "viable" in a narrow sense: "a person would likely be able to carry out plans made with the information provided". Whether an attack would succeed still depends on things the chatbots did not cover.
- GAO left out its prompts and techniques, summarizing "rather than providing details."
- It withheld "details on suggested targets, and specific materials needed for attacks."
- Anthropic asked GAO to list the techniques its model resisted. GAO declined, "to prevent potential malicious use of our work."
- OpenAI suggested removing or generalizing details to avoid misuse, and GAO says it "addressed all the safety-related items."
What OpenAI, Anthropic, Google and SpaceXAI Told GAO
All four companies commented in writing on a draft. The quotes below are GAO's summaries of their letters. Anthropic said it designs safeguards "on the assumption that any single protection can be defeated". It also said the study "describes the system as it was in May 2026, and it has continued to expand safeguards since".
Anthropic also asked GAO to state that there were too few trials for statistical significance. GAO agreed: "We acknowledge that our tests reflect a point in time and are not a statistically significant sample". It added that "the models all eventually provided information that could be used by malicious users".
OpenAI said ChatGPT refused many of GAO's requests or gave "safer alternative responses". It said the report blurs what came from ChatGPT and what the investigators added. GAO replied that its testers did use their own experience and the internet, which is what "a malicious actor using an AI platform as a tool" would do.
Google said the safety of its AI products "is never complete" and its safeguard work is "dynamic and improving". SpaceXAI said it would keep improving "safety and moderation techniques with each model release" and asked for transcript logs. In September 2026, GAO gave Anthropic and SpaceXAI the test accounts so they could find the chats.
- GAO agrees with Anthropic that because each chatbot got one random scenario, the results "cannot be compared with one another."
- The report makes no recommendations. Instead of recommendations, it has a section called "Concluding Observations."
- GAO also writes that "Every safeguard used in the industry is vulnerable to jailbreaks" (tricks that get around a chatbot's rules), citing a June 2026 Anthropic statement. Anthropic's own wording is narrower: it said every safeguard is vulnerable to "non-universal jailbreaks". This is background GAO cites, not a result of its tests.
- GAO also notes that "no AI systems, generative or otherwise, can be fully secured," citing the US standards agency NIST and others.
- AP reports that Anthropic said it is not able to verify GAO's findings, and that Google said GAO's testing occurred on earlier models.
What the GAO Report Means for Everyday AI Users
The report says some AI developers watch conversations as a safety step. It lists "monitoring and logging of user activity", "a combination of automated systems and human review", account suspension and "escalation to law enforcement". So some chats may be read by people at companies that use human review.
The same report repeats GAO's earlier privacy warnings. A chatbot may be unable to "forget" something a user wants deleted. Sensitive information typed into a prompt "could be stored and misused or aggregated with other information in the future".
- These were the spring 2026 versions, and the companies generally say they will keep improving their safeguards, so today's versions may behave differently; treat the report as a snapshot, not a ranking of today's apps.
- Check anything important a chatbot tells you against a second source, because a confident tone says nothing about accuracy.
- Check each app's settings to see whether your chats are used for training; our explainer on AI privacy risks has more on stored chats.
- Leave names, account numbers and other personal details out of prompts where you can; our guide on keeping your data safe covers the basics.
Elephas is a private AI app for your own documents, and its privacy features speak to the stored-chat worry above. Your files stay on your computer, and every answer cites the page it came from.
Before a question leaves your computer, Elephas hides personal details such as names, email addresses and phone numbers. With Elephas AI, your question and passages are not stored, not used to train AI models, and not shared with the model makers. Or pick a model that runs on your computer, so nothing is sent.
What GAO Concluded
In its Concluding Observations, GAO writes: "The details behind the information provided by the AI platforms were not novel. Many of the specifics are available on the internet."
GAO adds that "the generation of plans that could have taken weeks or months were instead completed in hours with the assistance of AI." It says the platforms "could also raise ideas or methods that would not have been readily surfaced by basic research."
- For private AI on your own documents, Elephas is free for 7 days, no card needed.






