A homeowner’s furnace quits on a Sunday. Two years ago they searched. Now they open ChatGPT and type “who should I call for emergency furnace repair in Winnipeg”. They get three names and a short reason for each. They call the first one. The other eleven HVAC companies in the city never existed as far as that customer is concerned, and none of them will ever see the search that skipped them.
Getting named by an AI assistant is three separate problems, and they have to be solved in order: whether the AI can read your site at all, whether your site says anything worth quoting, and whether anyone else on the internet vouches for you. Almost everything written about this skips straight to the third, or sells you a checklist for the second, while the first one is quietly failing.
The first one is binary and it is the one nobody checks. I know that because we shipped it ourselves. We sell AI search, and from 22 to 31 July 2026, our own website blocked every major AI crawler. Nine days. Nobody at Nord chose it in any way we could reconstruct, and nothing inside WordPress showed it.
Here is the number that makes this worth half an hour of your time. On Calculated Moving, a Winnipeg moving company we run SEO for, the AI assistant channel produced 13 lead-form submissions from 46 sessions between 1 January and 31 July 2026. That is 0.28 percent of the site’s traffic producing 0.74 percent of its leads, with visits lasting 2 minutes 8 seconds, roughly double the site average. Small. Real. And it did not exist at all before 15 May: Analytics returns no data for that channel across the first 135 days of the year, then 46 sessions and 13 leads in the eleven weeks after.
Key takeaways
- Three problems in strict order: access, then a worth-quoting answer, then reputation. Fixing the second and third while the first is broken buys you nothing.
- Access is not visible from your website. Ours was blocked at Cloudflare while WordPress showed a perfectly clean robots.txt for nine days.
- Test it with a user agent request, not by reading your site. Nothing on the page tells you a crawler was turned away.
- Cloudflare’s managed robots.txt blocks Amazonbot, Applebot-Extended, Bytespider, CCBot, ClaudeBot, Google-Extended, GPTBot and meta-externalagent in one switch.
- From 15 September 2026, new domains onboarding to Cloudflare get Training and Agent crawlers blocked by default on pages that display ads. Search stays allowed.
- Blocking Google-Extended does not hurt your Google rankings. Google states it is not a ranking signal and does not affect inclusion in Search.
- Being crawled is not being cited. Six AI crawlers reached our site within a day of the fix, and that is the start of the work, not the result.
- What gets quoted is the answer nobody else can write: your numbers, your jobs, your judgement.
- AI assistant traffic is under one percent of sessions on the client sites we measure, and converts well above site average. Plan for both halves of that sentence.
Layer one: can an AI read your site at all
This is the only part of AI search that is genuinely binary. Either the crawler gets your page or it does not.
There is no partial credit, no gradual improvement, and no amount of content strategy that survives a 403.
It is also invisible. Your site looks perfect to you, because you are a person using a browser. The block, when there is one, is aimed at something else entirely.
What happened to us
On 22 July we ran a user agent test against our own domain as part of building a service page about this exact topic. GPTBot, ClaudeBot, CCBot and Google-Extended were all being turned away. Our origin robots.txt, the one WordPress generates, was completely clean. It had been clean the whole time.
The block was at Cloudflare, sitting in front of the site, in a feature called AI Crawl Control. One setting called “Managed robots.txt” was on. Cloudflare’s own documentation describes what that does: it “generates and maintains a robots.txt file that instructs known AI crawlers to stay away from your content”, covering Amazonbot, Applebot-Extended, Bytespider, CCBot, ClaudeBot, Google-Extended, GPTBot and meta-externalagent. And critically, if your site already has a robots.txt file, Cloudflare “will prepend our managed robots.txt before your existing robots.txt, combining both into a single response”.
Prepend. Your file is still there, untouched, and completely irrelevant. That is why nothing in WordPress showed a problem. A separate firewall rule was returning 403 to ClaudeBot outright.
The setting is not on by default. Cloudflare’s documentation is clear that it has to be activated. Which means at some point somebody with dashboard access turned it on, or accepted it during onboarding, and it was never revisited. That is the honest version and it is worse than a conspiracy, because it is the version that can happen to you.
Nine days, on the site of a company that sells this service, with a homepage that says be the answer when customers ask AI.
Test yours in two minutes
Do not read your site. Ask for it the way a crawler does, and watch what comes back. On a Mac or Linux terminal:
curl -s -o /dev/null -w "%{http_code}n" -A "GPTBot/1.2" https://yourdomain.ca/
curl -s -o /dev/null -w "%{http_code}n" -A "ClaudeBot/1.0" https://yourdomain.ca/
curl -s -o /dev/null -w "%{http_code}n" -A "PerplexityBot/1.0" https://yourdomain.ca/
You want 200 from every one. A 403 is a firewall refusing them. Then read your live robots.txt in a browser at yourdomain.ca/robots.txt and look at the very top of the file, above anything you recognise. Managed directives get prepended, so the block will be sitting above your own rules, not mixed in with them.
If you are on Cloudflare, the setting is under AI Crawl Control, and there are two places to look: the Signals page for the managed robots.txt toggle, and the Security page for per-crawler blocks. If you are not on Cloudflare, the same job is usually done by a security plugin or your host’s bot protection.
This is the same family of problem as a website Google cannot index, and it is worth running both checks in the same sitting, because the causes overlap and the symptoms look nothing alike. If you want the longer walkthrough of the crawler side specifically, we wrote one: your website might be blocking AI.
What happened after we fixed it
We turned the managed robots.txt off on 31 July. Within twenty four hours, Cloudflare’s own dashboard recorded these requests:
| Crawler | Operator | Requests allowed | Failed |
|---|---|---|---|
| Googlebot | 20 | 0 | |
| Claude-SearchBot | Anthropic | 12 | 0 |
| BingBot | Microsoft | 11 | 0 |
| PerplexityBot | Perplexity | 6 | 0 |
| CCBot | Common Crawl | 1 | 0 |
| Claude-User | Anthropic | 1 | 0 |
| ClaudeBot | Anthropic | 1 | 0 |
| GPTBot | OpenAI | 1 | 0 |
Six AI crawlers turned up in a day. They had been trying and being turned away, and the moment the door opened they came through it.
Read that table honestly, though. Those are crawl requests. Not citations, not traffic, not a single booked job. It is the difference between the shop being open and the shop having customers.
The one worth knowing about, before September
Cloudflare’s classification of crawlers is worth understanding because it changes what “blocking AI” even means. In its 1 July 2026 announcement Cloudflare split bots into three jobs: Search, “any behavior that collects or indexes your content, so it can answer questions about it later”; Agent, “automated behavior that is acting, usually in real time, on a person’s behalf, to get something done right now”; and Training, “a crawler taking your content to train or fine-tune a model”.
Those are three different decisions and most owners make all three with one switch. Blocking training while keeping search is a perfectly reasonable position. Blocking search is how you disappear from the answer.
From 15 September 2026, for all new domains onboarding to Cloudflare, Training and Agent are blocked by default on pages that display ads, and Search stays allowed. Cloudflare’s reasoning is that “an ad is a signal that a website owner meant for a person to land there and see it”.
Two honest qualifications, because this is where the commentary gets it wrong. It applies to new domains onboarding, not retroactively to every existing site. And it is scoped to pages that display ads, which most trades and home-service sites do not have. If that is you, this default will probably never touch you. Know it exists, check your own settings before September anyway, and do not let anyone sell you a panic package over it.
One more thing worth knowing, because the same announcement flags it: multi-purpose crawlers such as Googlebot, Applebot and BingBot will be blocked for customers who have chosen to block Training. Those crawlers do both jobs with one agent. Block the training half and you lose the search half with it.
Layer two: give it something only you could have written
An assistant quotes the page that supplies something the other eleven results did not, which in practice means your own numbers, your own prices and your own cases.
Once the door is open, the question changes completely. Now it is not “can it read me”, it is “why would it quote me rather than the other eleven”.
Here is Daniel’s answer, and it is the reason this post exists in the form it does:
If you answer a question your potential client asked an AI platform with a unique answer from your own experience, you have more chance of getting cited.
Look at what an assistant is doing when it names three companies. It is assembling an answer from sources. A page that restates what forty other pages already say adds nothing to that answer, so there is no reason to reach for it. A page carrying a number, a job, a mistake or a decision that exists nowhere else is the only thing that can make the answer better.
That is not an AI trick. It is the same logic Google has run on for years, which is why the two jobs converge rather than compete.
In practice, for a trades business, the things that qualify are unglamorous and you already own them:
- What the job actually costs in your city, in dollars, this year. Not “it depends”.
- How long the work takes, and what makes it take longer.
- The failure you see most often and what causes it.
- What you refuse to do, and why. This one gets quoted far more than people expect.
- A real job with real numbers and a real outcome.
Everything on that list is a thing you know and a competitor’s copywriter does not.
This post is the demonstration. The nine days above are not a marketing anecdote, they are the only version of that story that exists, and I would rather publish our own failure than another checklist. If this piece gets cited by anything, that is the part that will do it.
The structural half is simpler than the industry pretends. Answer the question in the first sentence of the section, before the context, so the answer can be lifted out on its own. Use a heading that matches what a person would actually ask. Keep one idea per paragraph. That is most of it. Anything past that is diminishing returns compared with having something to say.
We build pages this way as part of SEO in Winnipeg and every site we design and build, because a page structured for a person to read and a page structured to be quoted turn out to be the same page.
Layer three: the part you cannot write yourself
An assistant does not only read your site. It reads what other places say about you, and it weighs agreement between them.
Daniel’s read, and ours from watching this across client accounts, is that reviews and mentions are the compounding layer. Slower than a config change, and worth more over time.
The practical shortlist is short and it is not exciting:
- Reviews, in volume, with recent dates and real detail. They are also the strongest local ranking signal you have, so this is not a separate project. Our guide to responding to Google reviews covers the half of this most owners skip.
- A Google Business Profile that is complete and correct. If you are not appearing there, that is its own diagnosis and it comes before this.
- Consistent name, address and phone number everywhere. Disagreement between sources is a reason for an assistant to name someone else instead.
- Being mentioned by name in places that are not yours. Local press, suppliers, associations, a trade body listing. A mention without a link still counts here, which is the part that surprises people who learned SEO in the link-building era.
None of this is fast. All of it also does something for your normal search rankings, which is the argument for doing it at all.
What about llms.txt
We publish one at nordmarketing.ca/llms.txt, it is a plain-text summary of the site aimed at language models, it costs nothing to maintain, and we are not going to tell you it is why anyone gets named.
No major AI provider documents it as a requirement in its published crawler guidance. Every one of them points at robots.txt for access control, which is layer one, which is the thing that actually decides whether you are readable. Treat llms.txt as cheap and unproven, do it if it is free, and be suspicious of anyone selling it as the centre of a strategy. We will write up what it is and who reads it separately, because it deserves a straight answer rather than a paragraph.
The same caution applies to the wave of AI-readiness audits that check your schema markup and your llms.txt and never once request your homepage as a crawler. That is the enemy in this whole category: a checklist sold against a site that is blocked at the edge, where every item passes and the answer is still no. What we do instead is boring and takes two minutes. We ask for the page as GPTBot and see what comes back, before anything else gets discussed.
Name the ceiling
Four things this will not do.
It will not make you a large channel this year. On the two client properties we measured for this post, AI assistant sessions were 0.28 percent and 0.17 percent of total traffic. Anyone showing you a deck where this is the main event is showing you someone else’s business.
Being crawled is not being cited. We have eight crawlers reaching our site and no claim to make about how often we get named, because measuring that properly is still genuinely hard. That got one step easier on 3 June 2026, when Search Console started counting impressions inside AI Overviews and AI Mode. We pulled that report across four client sites and wrote up what 25,000 AI impressions actually look like, including the pages it favours and the clicks it does not produce. Anyone quoting you a precise AI citation share should be asked exactly how they counted it.
It will not beat a better-reviewed competitor. Layer three is the slow one for a reason. If they have 180 reviews at 4.9 and you have 11 at 4.2, no amount of layer one and two closes that gap this quarter.
Blocking training is a real choice with a real cost, and it is yours. We are not going to tell you to allow everything. We will tell you that blocking Google-Extended is the cheap end of that decision, because Google’s own crawler documentation states plainly that “Google-Extended does not impact a site’s inclusion in Google Search nor is it used as a ranking signal in Google Search”. You can opt out of Gemini training and keep every bit of your search visibility. Blocking Search-class crawlers is the expensive end, and that is the one people do by accident.
The counter-argument, taken seriously
There is a reasonable position that says none of this matters yet in a city the size of Winnipeg, and you should spend the hour on reviews instead.
The volume numbers support it. What moves it is the shape of the traffic. On Calculated Moving, AI assistant visits ran 2 minutes 8 seconds, roughly double the site average, and engagement was roughly half again above the site’s own rate. On Profixit, the Winnipeg appliance repair company we run search for, those sessions engaged at 91.67 percent against a 77.94 percent site average between 1 March and 31 July 2026. Two unrelated trades businesses, same pattern: tiny volume, unusually committed visitors. It reads like people arriving already sold, because something did the selling before the click.
The timing is the other half of it. Calculated Moving’s AI assistant traffic was not small before May, it was zero, for 135 consecutive days. Then it started and did not stop. A channel that goes from nothing to thirteen leads in eleven weeks is not yet a big channel, and it is not one you want to meet for the first time in a year.
Two sites is a weak signal and I will call it that. But layer one costs two minutes to check and either zero or one afternoon to fix, and the channel went from nothing to thirteen leads on one site in three months. That is a good trade at that price, and it is a bad reason to reorganise your whole marketing plan.
The two-minute version
Run three curl requests against your own domain, and if any one of them is not a 200 then nothing else on this page matters until it is.
- Run the three
curlcommands above against your own domain. Anything that is not200is your answer and nothing else matters until it is fixed. - Open
yourdomain.ca/robots.txtand read the top of the file, above your own rules. - If you are on Cloudflare, check AI Crawl Control, both the Signals page and the Security page.
- Decide Search, Agent and Training separately, on purpose, and write down what you chose.
- Then, and only then, go and write the page only you could write.
If you want a second pair of eyes, send me your domain and I will run the crawler test and tell you what comes back, in plain terms. Nothing to sign. If the answer is that your site is wide open and your problem is that you have nothing quotable on it, I will tell you that instead, because that is the more common answer and it is the more expensive one to fix.
Questions owners actually ask
How do I get my business to show up in ChatGPT?
Check three things in order. First, that AI crawlers can reach your site at all, by requesting your homepage with a crawler user agent and confirming you get a 200 rather than a 403. Second, that your pages answer real questions with information only you have, such as your own pricing, timelines and jobs. Third, that other sources agree you exist and are good, mainly reviews, a correct Google Business Profile and mentions elsewhere. Skipping to the third while the first is broken produces nothing.
How do I check whether my website is blocking AI crawlers?
Request your homepage with the crawler’s user agent string and look at the status code, because nothing on the page itself will tell you. A 403 means a firewall is refusing it. Then read your live robots.txt in a browser and check the very top of the file, because managed rules from a CDN are prepended above your own and will not appear in your website’s own settings.
Why does my robots.txt look fine but AI still cannot read my site?
Because the file you are looking at may not be the file being served. Cloudflare’s managed robots.txt feature prepends its own directives before your existing file and serves both as one response, so your origin file stays clean and completely irrelevant. Separately, a firewall rule can return 403 to a specific crawler regardless of what any robots.txt says.
Does blocking Google-Extended hurt my Google rankings?
No. Google’s crawler documentation states that Google-Extended does not impact a site’s inclusion in Google Search and is not used as a ranking signal in Google Search. It controls whether your content can be used to train Gemini models and for grounding in Gemini Apps and Vertex AI. You can decline the training use and keep your search visibility.
What is changing on Cloudflare in September 2026?
From 15 September 2026, new domains onboarding to Cloudflare will have Training and Agent crawlers blocked by default on pages that display ads, while Search crawlers stay allowed. It applies to new domains rather than retroactively to every existing site, and it is scoped to ad-displaying pages, which most trades and home-service sites do not have. Check your own settings regardless.
What is the difference between Search, Agent and Training crawlers?
Cloudflare defines Search as behaviour that collects or indexes content so it can answer questions about it later, Agent as automated behaviour acting in real time on a person’s behalf, and Training as a crawler taking content to train or fine-tune a model. They are three separate decisions. Blocking training while allowing search is a common and reasonable position, but many owners block all three by accident with a single switch.
Do I need an llms.txt file to be cited by AI?
There is no evidence that you do. No major AI provider documents llms.txt as a requirement in its published crawler guidance, and all of them point at robots.txt for access control. It costs nothing to publish one, so publish it if you like, but treat any strategy built around it with suspicion.
How much traffic does AI search actually send a local business?
On Calculated Moving and Profixit, measured through to 31 July 2026, the AI assistant channel was 0.28 percent and 0.17 percent of total sessions. It is under one percent. On the larger of the two it produced 13 lead-form submissions from 46 sessions in seven months, and those visits ran roughly twice the site’s average engagement time, so it converts well above its size. Both halves of that are true and you should plan for both.
Is this different from normal SEO?
Less than the industry implies. The access layer is genuinely new, and it can fail in ways classic SEO audits never check. The other two layers, saying something worth quoting and being vouched for by other sources, are what search has rewarded for years. If someone is selling AI optimisation as a wholly separate discipline with a separate retainer, ask them which specific parts do not overlap.
When did AI Overviews reach Canada?
Google brought AI Overviews to Canada on 28 October 2024, and launched AI Mode in Canada in August 2025. If your reference point for what a Google results page looks like is older than that, it is worth searching a few of your own service terms and seeing what sits above the results now.
