Skip to content

Uncategorized

We Went Looking for Lies About Our Own Pilot

July 28, 2026 · 7 min read · Ignite Agent

Illustrative grid of claim dots with a few flagged in amber, representing a monitor watching for false or poisoned claims about a brand
Illustrative graphic, for visual presentation only. Generated for Ignite.

Every story about AI visibility is really a story about what a model says when you are not in the room. So this summer we did something uncomfortable. We pointed our own tooling at our first pilot brand, and we went looking for lies.

Not typos. Not stale copy. Lies: false or corrupted claims an AI answer might repeat about a real business to a real buyer who is deciding whether to trust it.

Can an AI be made to repeat false claims about a brand without a breach?

Yes, and it does not require access to anything you own. When someone asks a modern AI assistant about a company, the answer is often assembled in the moment from documents the engine retrieves right then. Retrieval is the soft spot.

A retriever scores each candidate passage by how closely its embedding matches the embedding of the question, then hands the top few to the model to answer from (Karpukhin et al., Dense Passage Retrieval, EMNLP 2020[1]). Nothing in that step checks whether a passage is true. It checks whether it looks relevant.

That is the opening. If an attacker publishes documents crafted to look highly relevant to a question a buyer would ask about your brand, those documents can be the ones retrieved, and the answer can be assembled from them. This is not a separate exploit; it follows directly from the retrieval step above, where scoring rewards apparent relevance and never checks truth[1]. The uncomfortable part is what it does not require: no break-in, and no access to anything you own. It requires publishing.

Catching that is exactly what Ignite’s Claim and Poison Monitor is built to do. It samples the answers customers actually get on a weekly cadence, so a poisoned claim surfaces as a bounded signal rather than a hunch.

What does Ignite’s Claim and Poison Monitor do?

The Claim and Poison Monitor extracts every claim a brand makes about itself and checks each one against the answers AI assistants actually give, on a weekly cadence rather than once. On 2026-07-26 we brought it live on our first pilot and ran it against real data. It extracted and checked 95 claims the brand makes about itself[2]. We built it to run often rather than once, because a lie you catch a month late has already done its work.

The point of that first pass is not a headline. It is a baseline, and it is our own pilot measurement, not a general result. Before you can catch a lie about a brand, you have to know what the brand actually asserts, in its own words, so that drift, a contradiction, or an injected falsehood has something honest to be measured against. Those 95 checked claims are that starting map, and a number stays out of the report until it has at least 2 samples behind it.

We look for this where it matters: the answers a real customer would actually get when they ask about the brand, not a sanitized internal test. Over a rolling 21-day window the probe accumulated 1,658 AI answer samples for the pilot[3]. We will be honest about what that number is. It is our first pilot: single-tenant, point-in-time, and not a benchmark anyone should expect to reproduce.

Can you make the true version of a claim easier for an AI to find?

You can, up to a point. Structure-aware, clearly written content is retrieved far more reliably than arbitrarily split text: in a retrieval-augmented generation (RAG) pipeline lab test, structure-aware paragraph chunks reached precision@1 of about 24% against 2 to 3% for arbitrary character splits, and the effect held across multiple embedding models (Shaukat et al., chunking study, 2026[4]). That is a lab result about a retrieval pipeline, not a measurement of any specific frontier engine, and not a number your brand will see.

Making claims visible and quotable helps too. Adding statistics, citations, and quotations to the content an engine can see lifted answer visibility by 30 to 40% on the GEO (generative engine optimization) benchmark and by 22% Position-Adjusted Word Count (Aggarwal et al., GEO: Generative Engine Optimization, KDD 2024[5]). Read that precisely. Visibility there means Position-Adjusted Word Count, how much of your own text the answer quotes back. It is not traffic, ranking, sales, or brand awareness, and it is not a result you or our pilot should expect to get. It measures whether your words get used, not whether you win.

Even getting retrieved is not the end of it. Models do not use every passage in their context equally: text placed at the start or end of a long context tends to be used well, while material in the middle degrades (Liu et al., Lost in the Middle, TACL 2024[6]). Being present is not the same as being read.

What can a claim monitor actually promise?

A claim monitor can raise the floor. It cannot guarantee the truth wins. Structuring a brand’s claims clearly, keeping entity names consistent, and making the true version easy to retrieve all raise the odds the honest passage is the one retrieved, the same precision@1 edge structure-aware chunks showed over arbitrary character splits (Shaukat et al., chunking study, 2026[4]). That raises the cost of a casual falsehood and lowers the odds it takes hold.

What we cannot honestly promise is that clean, well-structured truth automatically overwrites a determined lie. An attacker optimizing an injection against one specific question a buyer asks is a hard adversary. Good structure raises your floor and raises their cost. It does not end the fight, and we are not going to tell you it does.

That is the whole reason a monitor exists. You do not set the record straight once and walk away. You watch the answers customers actually get, on a weekly cadence, you catch the drift, and you respond. We would rather tell you that than sell you a guarantee we cannot keep.

This is chapter two of what we are building. We went looking for lies about our own pilot because that is the honest way to find out whether the defense works before we point it at yours.

References

  1. [1] Karpukhin et al., Dense Passage Retrieval, EMNLP 2020: a retriever scores each passage by the embedding similarity between the question and the passage and returns the top-k. Scope: the retrieval mechanism, cited to explain why crafted content can be retrieved and why a well-structured true version can be too. https://arxiv.org/abs/2004.04906
  2. [2] Our first pilot, Claim and Poison Monitor: 95 claims the brand makes about itself, extracted and checked on 2026-07-26. This is our own pilot measurement, a single-tenant baseline, point-in-time, not a general result.
  3. [3] Our first pilot, AI answer samples: 1,658 answer samples accumulated over a rolling 21-day window. This is our own pilot measurement, single-tenant and point-in-time, not a benchmark anyone should expect to reproduce.
  4. [4] Shaukat et al., chunking study, 2026 (arXiv 2603.06976): structure-aware paragraph chunks reached precision@1 of about 24% versus 2 to 3% for arbitrary character splits, holding across multiple embedding models. Scope: a retrieval-augmented generation (RAG) pipeline lab result, indirect evidence about any specific frontier engine, not a customer’s own number. https://arxiv.org/abs/2603.06976
  5. [5] Aggarwal et al., GEO: Generative Engine Optimization, KDD 2024: adding statistics, citations, and quotations to visible content lifted answer visibility 30 to 40% on the GEO benchmark and 22% Position-Adjusted Word Count. Scope: visibility is Position-Adjusted Word Count, how much of your own text the answer quotes back, not traffic, ranking, sales, or brand awareness, and not a result a reader or our pilot will get. https://arxiv.org/abs/2311.09735
  6. [6] Liu et al., Lost in the Middle, TACL 2024: a U-shaped positional-use curve, where content at the start and end of a long context is used well and material in the middle degrades. Scope: a qualitative finding about how models use their context, with no headline percentage. https://arxiv.org/abs/2307.03172

See what AI says about your business.

Run a free scan and find out whether AI names you when your customers ask.

Run my scan