Last week, the single most active crawler on our site was ClaudeBot, Anthropic's. Eighty-eight visits, more than Google's crawler, more than OpenAI's. By raw attention, Claude is our most devoted reader.
So this week we asked Claude our eight standard buyer questions, the ones a founder would ask before hiring an AI staffing agency like ours. We ran each question three times, twenty-four samples in all. Claude named us zero times. Not wrong about us, not unflattering. Absent.
This piece is about the afternoon we spent finding out why, because the answer turned out to be a piece of plumbing I have not seen a single AI-optimization guide mention, and you can check it on your own site in about five minutes.
The five-minute diagnostic
When an engine never cites you, there are three layers to check, in order: is your content citable, is it retrieved, is it indexed at all. The guides spend all their ink on the first layer. The problem is usually further down.
Claude's web search does not run on some private Anthropic index of the web. It is powered by Brave Search, the independent index built by the browser company. When Claude searches on your behalf, Brave picks the candidates. So we checked Brave directly. Our brand ranked in none of the top 10 results for any of the 7 buyer questions we could find. Then the blunter test: site:coworkers.global on search.brave.com. Nothing. Multiple variations. Nothing.
Our site is not losing in Brave's index. It has never been in it.
The reflex is to blame robots.txt. Ours allows every crawler, and the logs prove they come. That is what makes the finding worth writing down. You can be crawled daily by an AI company's training bot and still be invisible to that same company's search index, because they are different systems fed by different pipes. Anthropic runs three separate crawlers: one for training data, one that fetches pages a user asks about, and one that builds the search index. Over six weeks the training crawler hit us 743 times and the page fetcher 71. The search crawler, the one that would put us in the index Claude actually searches, hit us zero times. Our logger does track that crawler by name, so the zero is a real absence and not a gap in the instrument.
Every engine is wearing another engine's index
The general lesson is the one that changed how I read this whole category. An answer engine's citations come from its retrieval backend, which is usually someone else's index. Copilot retrieves only from Bing: if Bing has not indexed your page, Copilot cannot cite it no matter how good the page is. Claude retrieves from Brave. Gemini and Google's AI answers retrieve from Google. ChatGPT's search is widely reported to rely on Bing as well, though OpenAI does not confirm this.
So "optimize for AI search" decomposes into a question almost nobody asks: which indexes am I actually in? We ran that audit on ourselves this week, and it was humbling. Google, yes. Bing, verified and submitting since early July, though I have not yet read the crawl report that would tell me whether the pages are actually in. Brave, no. So one major answer engine could not have cited us this year regardless of what we published, and a second is an open question I cannot yet answer. Every content dollar aimed at those engines was spent above a pipe nobody had checked.
I want to be precise about the emotional register here, because the vendors selling AI-visibility dashboards will read this as a pitch for more measurement. It is the opposite. No dashboard we evaluated would have told us this. The diagnosis took a site: query on a search engine most marketers have never opened.
What you can actually do about it
Brave has no webmaster console. There is no form where you submit your site and wait. Its index discovers pages through its own crawling, links from pages it already indexes, and real visits by users of the Brave browser, which feed into an anonymized discovery system. That constrains the playbook to three moves.
First, earn links from surfaces the index already trusts. In our seven-question read, Reddit appeared in Brave's top ten for every single question. Not most. Every one. Whatever you think of Reddit, the index thinks highly of it, and a legitimate, useful mention with a link creates the crawl path that a young site cannot create for itself.
Second, fix the parallel problem when a console exists. Bing Webmaster Tools is free, takes an hour to set up properly, and since February 2026 includes the first native AI-citation report from any platform. Its Crawl Information report answers "is Bing crawling me" directly, retroactively, and for nothing. We are enrolled, and that report is the next thing I am reading.
Third, wait. Index inclusion for a young site is partly a function of time and accumulating links, and no amount of on-page optimization accelerates a pipe you are not connected to.
The zero that wasn't
One correction, because it is the most useful thing I learned this week. My first draft of this post said our Bing crawler traffic was zero. It was not. Our own crawler logger matches incoming requests against a hardcoded list of AI bot names, and Bingbot was never on that list, so six weeks of silence in the log carried no information at all. I had read a zero off an instrument that could not have shown anything else.
That is the second lesson, and I think it may be the better one. Check the plumbing before the message. Then check the instrument before you trust the zero.
Current view, subject to change
My current view is that index inclusion is the zeroth step of AI optimization, before content, before schema, before any paid tool, and that it belongs at the top of every AIO audit as a pass-fail gate. We have rewritten our own audit method to check it first, because a failed inclusion check makes every downstream lever moot. What would change my mind: if the engines converge on shared or open indexes, or if Anthropic's own search crawler grows into the primary path and Brave stops mattering. There are early signs of the second, and we are watching our logs for it.
Final thoughts
Post one of this series admitted our share of voice was near zero after six weeks of trying. This post is what finding out why looks like: not a clever content insight, a missing pipe. I find that oddly encouraging. Invisible-because-unindexed is a fixable condition with a known mechanism, which beats invisible-for-reasons-unknown on every axis that matters.
The broader habit is the real takeaway. When a channel shows zero, check the plumbing before the message. The message was fine. The pipe was never connected.
Regards,
Charles Stack