sakutto
Generative AI

LLM Poisoning: A Fake Think Tank Built for AI

AI searchInformation operationsGenerative AI
LLM Poisoning: A Fake Think Tank Built for AI

The fake think tank behind this LLM poisoning case

Understanding the technique is faster with the artifact in front of you, so start with what was found.

The Hanover Institute is not a real research body

The Hanover Institute for Public Policy presents as a new research organization working on Israel and Palestine, publishing think-tank-shaped reports with titles like whether AIPAC uses dark money in elections. According to the reporting, it is not a real think tank. Alice Lee, an analyst at NewsGuard, a disinformation tracking company, describes it as an exact imitation of a credible American think tank — the generic name, the site layout, and the red-white-blue palette included.

View official source →
"But the Hanover Institute is not a real think tank." (the finding)/"It's a perfect mimicry of a typical credible American think tank, right down to the generic name, the site layout, and the red-white-blue color scheme," (NewsGuard analyst comment)— from Responsible Statecraft's investigation

No bylines, and the disclosure sits in small print

Two tells are named in the reporting. None of the reports carry a byline. The other is a small disclaimer at the bottom of the page, which notes the organization was created on behalf of the Israeli Government Advertising Agency by Piro, Inc. That firm has received $900,000 from the Israeli government for the work, subcontracted through Havas Media, a French public relations conglomerate. Publishing started on August 6, and the count passed 100 reports by the August 17 report date.

View official source →
"None of the reports have bylines. A small disclaimer at the bottom of the webpage notes that the organization was created on behalf of the Israeli Government Advertising Agency by Piro, Inc…" (bylines and the disclaimer)/"Piro, Inc, the firm that created the Hanover Institute, has received $900,000 from the Israeli government for its work. Like many other contractors working for Israel, Piro’s work is subcontracted through Havas Media, a French public relations conglomerate." (the contract)/"The fake think tank has churned out over 100 reports since it started publishing on August 6." (publication volume)— from Responsible Statecraft's investigation

Why AI chatbots are prone to citing it

The production details are the optimization. Nothing about the format is incidental.

Statistics, footnotes, and a neutral tone do the work

The reports come with footnotes and tables of contents, and they argue in a neutral register, which the reporting says helps them appeal to chatbots like Claude or Gemini. Models favor concrete statistics and data along with strong citations and sources, and these articles carry all of it, per the NewsGuard analyst. Many follow a formula, opening on the kind of plain question a person would actually type into a chatbot. One report is titled "What Caused the Displacement of Palestinians in 1948?"

View official source →
"The institute’s “data reports” have footnotes and tables of contents, and they present arguments in a neutral tone, helping them appeal to chatbots like Claude or Gemini." (report format)/"LLMs favor concrete statistics and data, as well as strong citations and sources, which these articles all have," (NewsGuard analyst comment)/"Many of the reports are formulaic, starting with an innocent question that someone might ask a chatbot." (report structure)— from Responsible Statecraft's investigation

"AI Story Optimization" is sold as a service

Piro's own website describes writing content engineered for how LLMs evaluate credibility, and gives that service a product name: AI Story Optimization. A co-founder pitched the capability on LinkedIn last month, describing how a question to ChatGPT, Gemini, or Perplexity about your category returns one confident paragraph, and how the firm spent months reverse-engineering how that paragraph gets assembled. The pitch treats a chatbot's answer as a surface that can be worked on, the way search results once were. The agreement Piro filed with the Department of Justice does not explicitly state that its work for Israel is meant to influence AI.

View official source →
"Piro’s website says that it “author(s) content engineered for how LLMs evaluate credibility,” describing this service as \"AI Story Optimization.\" Others refer to this practice of influencing artificial intelligence as “LLM poisoning.”" (Piro's site copy)/"When someone asks ChatGPT, Gemini, or Perplexity about your category, an answer comes back in one confident paragraph. Most brands have no idea how that paragraph gets built. So we spent months reverse-engineering it…" (co-founder's LinkedIn post)/"Piro does not explicitly state in its agreement submitted to the Department of Justice that its work for Israel is to influence AI." (the DOJ filing)— from Responsible Statecraft's investigation

An AI detector flagged 11 of 12 sampled articles

Responsible Statecraft ran 12 randomly chosen Hanover Institute articles through GPTZero, an AI detection tool that claims a low false-positive rate. Eleven came back as AI-written with high confidence, and the twelfth with moderate confidence. That result also fits the throughput: more than 100 reports in just over a week.

View official source →
"RS analyzed 12 random Hanover Institute articles using GPTZero, a popular AI detection software that claims a low false-positive rate. GPTZero flagged 11 of the articles as AI-written with “high confidence”; it flagged one article as AI-written with “moderate confidence.”" (GPTZero results)— from Responsible Statecraft's investigation

How to check where an AI answer came from

A reader's options are narrow here, but they are not empty.

Check the byline and who funded the site

An AI answer reveals nothing about its provenance unless you open the links it cites. However official an organization's name looks, whether a piece has an author and who pays for the site are both things you can verify yourself. Any site claiming to be a research institute should say somewhere who wrote the work and who funded it. When neither answer is on the page, the absence is itself an answer.

Copilot and Gemini have already ingested such sites

There is a documented case of chatbots picking material like this up. Responsible Statecraft reports that Israel also contracted former Trump campaign manager Brad Parscale, under a $46.5 million contract, to build pro-Israel websites engineered to influence chatbots. It cites a separate Drop Site investigation finding that Microsoft Copilot and Google Gemini had been successfully trained on data from those sites, and that other chatbots frequently cite them without flagging them as part of an influence operation. For how much traffic AI search now sends at all, see our look at AI search referrals measured at Shopify; for the related problem of authorship itself being faked, see AI byline misuse.

View official source →
"Israel has also contracted former Trump campaign manager Brad Parscale to create pro-Israel websites engineered to influence chatbots as part of a $46.5 million contract. A Drop Site investigation last month found that many chatbots, particularly Microsoft Copilot and Google Gemini, had been successfully trained on data from those websites. Other chatbots frequently cite those websites without flagging them as part of an Israeli influence operation."— from Responsible Statecraft's investigation

When you do follow a citation, pulling the page down with its structure intact keeps the things that live outside the body text — the byline slot, the footer disclosure — from disappearing in a copy-paste.

Free ToolURL to Markdown ConverterConvert any public web page URL to Markdown. Preserves headings, tables, lists, and links — perfect for LLM and RAG preprocessing, research notes, and archiving web articles.Try it now →

Summary

LLM poisoning uses the trappings of a think tank as an engineering spec for getting cited. The documented case here is more than 100 unsigned reports produced in just over a week, carrying the statistics, footnotes, and neutral tone that models reward. Because an AI answer arrives pre-assembled as a single paragraph, the material behind it is invisible from the reader's side. The heavier the decision resting on that answer, the more it is worth opening the sources it names and checking who wrote them and who paid for them. For now, that is the verification a reader still has.

FAQ

Q. What is LLM poisoning?
It is the practice of shaping the material AI chatbots draw on in order to move what they say. The reporting describes a firm that writes content specifically for how language models judge credibility and markets it as 'AI Story Optimization.'
Responsible Statecraft — Israel Creates Fake Think Tank In Likely Attempt To Dupe AI Chatbots
Piro’s website says that it “author(s) content engineered for how LLMs evaluate credibility,” describing this service as "AI Story Optimization." Others refer to this practice of influencing artificial intelligence as “LLM poisoning.” Responsible Statecraft — Israel Creates Fake Think Tank In Likely Attempt To Dupe AI Chatbots
Q. Why would an AI chatbot cite a site like this?
Because the material carries the signals models reward. An analyst at a disinformation tracking company says language models favor concrete statistics and strong citations, all of which these articles have, and that the site copies a credible American think tank down to its layout and colors.
Responsible Statecraft — Israel Creates Fake Think Tank In Likely Attempt To Dupe AI Chatbots
LLMs favor concrete statistics and data, as well as strong citations and sources, which these articles all have, Responsible Statecraft — Israel Creates Fake Think Tank In Likely Attempt To Dupe AI Chatbots
Q. Has any chatbot actually cited this kind of site?
Yes, according to the reporting. A separate investigation found Microsoft Copilot and Google Gemini had been trained on data from a different set of pro-Israel sites, and that many chatbots cite them without flagging them as part of an influence operation.
Responsible Statecraft — Israel Creates Fake Think Tank In Likely Attempt To Dupe AI Chatbots
A Drop Site investigation last month found that many chatbots, particularly Microsoft Copilot and Google Gemini, had been successfully trained on data from those websites. Other chatbots frequently cite those websites without flagging them as part of an Israeli influence operation. Responsible Statecraft — Israel Creates Fake Think Tank In Likely Attempt To Dupe AI Chatbots

Related Tools

Related Tool Categories

Articles