Can ChatGPT Read Your HVAC Website? A 5-Minute Test

Here is what each verified crawler does, straight from the operators' own documentation:

Two details in that table catch most site owners off guard. First, OpenAI's settings are independent of each other: you can block GPTBot (training) while allowing OAI-SearchBot (search), and OpenAI's documentation explicitly describes that split as a supported configuration. Second, Google runs no separate "AI crawler" for its Search AI features — AI Overviews eligibility rides entirely on ordinary Googlebot crawling and indexing. The Google-Extended token you may have read about controls AI training and grounding in some of Google's other systems (like Gemini); per Google's documentation, it is not the control for AI Overviews.

Anthropic's ClaudeBot also crawls at meaningful scale — Vercel's crawler study measured 370 million monthly ClaudeBot fetches on its network — but for an HVAC contractor, the OpenAI, Perplexity, and Google families are where homeowner queries actually happen today.

1. Read your robots.txt (60 seconds). Open yourdomain.com/robots.txt in a browser. Search the text for GPTBot, OAI-SearchBot, PerplexityBot, and ClaudeBot. A Disallow: / under any of those user agents blocks that crawler; a Disallow: / under User-agent: * blocks all of them (and Googlebot too). No AI names at all means robots.txt isn't your blocker — keep going, because most real blocks live one layer down.

2. Fetch your homepage as an AI crawler (90 seconds). In a terminal, run:

   curl -I -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" https://yourdomain.com/
   

   That user-agent string comes verbatim from OpenAI's crawler documentation. HTTP/2 200 means the request got through. 403, 503, or an HTML challenge page means something between the crawler and your server — usually the CDN — is refusing bot traffic even though your robots.txt says nothing. This mismatch is the single most useful finding the test produces, because a spoofed user-agent check approximates (not perfectly replicates) how bot-protection rules treat the real crawler.

3. View your page without JavaScript (90 seconds). Load your homepage, press Ctrl+U (view source), and search the raw HTML for your city name and a core service phrase like "AC repair." Present in the source: AI crawlers can read it. Absent — visible on screen but missing from the source — your content is rendered by JavaScript, and the JavaScript section below explains why that makes it invisible to every major AI crawler except Google's.

4. Ask ChatGPT to read a page (60 seconds). In ChatGPT (with search enabled), paste a specific service-page URL and ask it to summarize what the company offers. This exercises the ChatGPT-User fetcher end to end. An accurate summary is a pass. "I can't access this website" from a page that loads fine in your browser points to a CDN or firewall block — go to check 5.

5. Open your CDN or firewall dashboard (60 seconds). If your site sits behind Cloudflare or a similar service, find the AI-crawler or bot-management settings and read what they're actually set to. This is the layer that overrides everything else: your robots.txt can allow a crawler that your firewall then blocks. The next section covers why this specific misconfiguration became common in 2025.

Log the results somewhere. If checks 1–4 pass, your access layer is clean and your AI-visibility work moves to content and reputation. If any check fails, you've found a silent eligibility problem that no amount of content spending will route around.

The evidence that this is widespread comes from the AI companies themselves: Perplexity publishes step-by-step WAF whitelisting instructions for Cloudflare and AWS specifically because firewalls challenge its bots often enough to need documentation. When a search engine has to teach site owners how to un-block it, the blocking is not an edge case.

The common silent blockers, ranked by how often we'd expect them on a contractor site:

• CDN default-blocking — Cloudflare's post-July-2025 default, or the one-click block toggled at some point and forgotten.
• Bot-fight / WAF rules — generic bot protection that challenges any non-browser user agent, AI crawlers included.
• Blanket robots.txt rules — a leftover Disallow: / from staging, or an aggressive plugin configuration.
• Login walls and lead-gate overlays — a crawler can only fetch what an anonymous visitor can load.

What our scan of five HVAC marketing agencies found

Because this launch has no client data yet, we generated our own: on August 13, 2026, we fetched the live robots.txt file of five prominent agencies serving HVAC contractors. The finding surprised us — not one of the five declares a single AI-crawler directive. No GPTBot, no OAI-SearchBot, no PerplexityBot, no ClaudeBot, allowed or blocked:

Read that carefully: it does not mean these agencies are blocked, and it does not mean they're negligent — a robots.txt with no AI directives defaults to allowing AI crawlers. What it means is that the AI-access decision for these sites is being made implicitly, by whatever their CDN does — the exact layer Cloudflare flipped to default-block in 2025. In the marketing industry that sells AI visibility, the access layer is running on defaults. That's the state of play your HVAC site competes in, and it's why the five-minute test checks the CDN layer explicitly instead of trusting robots.txt silence. (Robots.txt files change; this table reflects the files as fetched on August 13, 2026.)

Google's own advice for appearing in AI features says the same thing in one line: ensure crawling is allowed in robots.txt, and by any CDN or hosting infrastructure. The second clause is the one that bites.

The scale of what's at stake is in the same dataset: GPTBot alone made 569 million fetches on Vercel's network in one month, against Googlebot's 4.5 billion. AI crawling is already a fifth of the volume of the machine that built the entire SEO industry — and the AI fifth can't see client-rendered pages.

For an HVAC site, the practical translation:

• Vulnerable: sites built as single-page apps or on page builders that inject service lists, city pages, or phone numbers client-side. The page looks complete to a human and nearly empty to GPTBot.
• Safe: server-rendered or statically generated pages, where the full text ships in the initial HTML. Check 3 of the five-minute test (view source) tells you which kind you have.
• Partial: some builders server-render the shell but load reviews, FAQs, or schema via script — the crawler sees your header and loses exactly the content AI engines cite.

The fix is architectural, not editorial: rebuild the rendering path so critical content ships as HTML. That's a standard requirement in every site we build for HVAC contractors, and it's worth demanding from any vendor — a site that fails view-source is failing five AI crawler families at once, silently.

Our honest framing, since this is the layer we work in: allowing GPTBot does not buy placement in anyone's answers, and nobody can guarantee ChatGPT recommends you — treat any agency promising that as a red flag. What allowing access does is keep you eligible across every surface: search answers today (OAI-SearchBot, PerplexityBot, Googlebot) and model knowledge over time (GPTBot). Blocking training bots removes a surface for essentially zero commercial benefit to a service business, unless you have a specific reason — proprietary pricing data, licensed content — to withhold it.

If you want the access layer audited and then built into actual AI-answer visibility — measured, not promised — that's the scope of our GEO service for HVAC companies.