How do I find out which funnel stage my content fails at?

Work backwards with evidence per stage: server logs prove crawling (look for GPTBot, ClaudeBot, PerplexityBot hits), engine-specific site checks and search tools indicate indexing, quoting your page's phrasing in prompts tests retrievability, and citation tracking measures the final stage.

What is the most common dropoff point?

Retrieval. Most reasonably maintained sites get crawled and indexed, but their passages lose to competitors at retrieval time — the content doesn't match prompt language, isn't self-contained, or lacks the facts engines want to quote. That is a writing problem, not an infrastructure one.

What Is the Crawl-to-Citation Funnel?

Last updated: July 5, 2026 · Menra Team

The crawl-to-citation funnel is the four-stage pipeline every page must pass before it appears as a source in an AI answer: crawled by the engine's bot, indexed into its retrieval corpus, retrieved for a relevant prompt, and finally cited in the generated answer. Diagnosing AI invisibility means locating which stage leaks, because each stage fails for different reasons and demands different fixes.

What breaks at each stage?

Stage	Evidence it happened	Typical failure
Crawled	Bot hits in server logs	robots.txt blocks, CDN bot rules, slow responses
Indexed	Page findable via the engine's search layer	Thin content, noindex signals, duplicate/canonical conflicts
Retrieved	Answer content echoes your page	Passages don't match prompt language; poor chunking; low authority
Cited	Your URL in the citations	Engine used you but credited a stronger co-source; linkless mention

How do teams instrument the funnel?

Crawl-stage truth lives in log-file analysis: filter access logs for AI user agents and verify they reach the pages that matter. Index-stage checks are engine-specific — retrieval-backed engines will surface an indexed page when asked directly about its exact topic. Retrieval is probed with tracer tests: unique phrasings from your page appearing in answers prove retrieval even without a citation. The cited stage is what citation tracking measures continuously. The funnel's power is prioritization — a site with zero GPTBot hits should fix bot access, not rewrite headlines, while a fully crawled site with no citations should fix extractability and evidence density per the GEO optimization playbook.

Example

A vendor's docs never appear in ChatGPT answers. Logs show OAI-SearchBot hitting only the homepage: the docs subdomain's robots.txt inherited a blanket block. One line change reopens the funnel's mouth — no amount of content rewriting would have worked before it.

Frequently asked questions

How do I find out which funnel stage my content fails at?: Work backwards with evidence per stage: server logs prove crawling (look for GPTBot, ClaudeBot, PerplexityBot hits), engine-specific site checks and search tools indicate indexing, quoting your page's phrasing in prompts tests retrievability, and citation tracking measures the final stage.
What is the most common dropoff point?: Retrieval. Most reasonably maintained sites get crawled and indexed, but their passages lose to competitors at retrieval time — the content doesn't match prompt language, isn't self-contained, or lacks the facts engines want to quote. That is a writing problem, not an infrastructure one.

Keep exploring

See how AI engines talk about your brand — track mentions across ChatGPT, Perplexity, Claude, Gemini and 5 more. Start with Menra