ParSEO
  • Blog
  • Pricing
Sign inStart your Day 0
  1. Home
  2. /Free tools
  3. /AI agent readiness checker

Free tool

AI agent readiness checker: can ChatGPT and Google read your site?

Type your website. ParSEO reads its robots.txt the way each crawler does, visits as ChatGPT's robot, and shows who is allowed in and whether the page is easy for an AI agent to use.

Runs on ParSEO, with a small daily limit. No sign-up.

ParSEO reads only your public pages and files, the same ones every crawler reads. It changes nothing.

What this checker shows

Every crawler that reads your site looks for robots.txt first and follows the group that names it, or the * group if none does. The checker reads your robots.txt by the same rules (RFC 9309) and shows, for each crawler, whether it may read your home page. It also tells you whether you have an llms.txt file and which sitemaps robots.txt lists.

Search crawlers, AI search crawlers and training crawlers

They do different jobs, so block them separately:

  • Search crawlers (Googlebot, Bingbot) decide whether you can appear in Google and Bing at all. Bing also feeds Microsoft Copilot.
  • AI search crawlers (OAI-SearchBot, PerplexityBot, Claude-SearchBot, and ChatGPT-User or Claude-User when a person asks the assistant to open a page) decide whether an AI assistant can read and cite your site in an answer.
  • Training crawlers (GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot) collect pages to train future models. Blocking them doesn't remove you from search or AI answers.

How to fix a block you didn't mean

Many sites copied a "block all AI" robots.txt in 2023 and 2024. Those often block AI search too. To stay out of training but be found in AI answers, give each AI search crawler its own group that allows it, and block only the training crawlers:

  • User-agent: OAI-SearchBot then Allow: /, and the same for PerplexityBot and Claude-SearchBot.
  • User-agent: GPTBot then Disallow: / if you don't want OpenAI to train on your pages; the same for ClaudeBot, Google-Extended and CCBot.

When the checker finds a block, it writes these lines for you to copy. In WordPress, an SEO plugin such as Yoast or Rank Math can edit robots.txt for you. On Shopify, edit the robots.txt.liquid template. Changes count the next time each crawler reads the file, usually within a day.

Where robots.txt stops

robots.txt is a request, not a lock: respectful crawlers follow it, and the companies above say theirs do. It also doesn't stop a page that is linked elsewhere from appearing in Google; for that you need a noindex tag. The website health check checks both.

Questions people ask

Does blocking GPTBot remove me from ChatGPT answers?

No. GPTBot collects pages for training. ChatGPT search reads pages with OAI-SearchBot, and ChatGPT-User when a person asks it to open a page, so those decide whether ChatGPT can cite you.

What does Google-Extended do?

It is a robots.txt token, not a separate crawler. Blocking it tells Google not to use your pages to train Gemini models. It doesn't change Google Search or AI Overviews, which use Googlebot.

I have no robots.txt. Is that a problem?

No. Without robots.txt every crawler may read the whole site. Many sites add one anyway to list their sitemap.

How often can I run the checker?

Twenty live checks a day from one connection, free. A site checked in the last hour shows its saved answer.

Sources

  1. Google Search Central: Introduction to robots.txt
  2. Google Search Central: Google's common crawlers, including Google-Extended
  3. OpenAI: Overview of OpenAI crawlers
  4. Perplexity: Perplexity crawlers
  5. RFC 9309: Robots Exclusion Protocol

Updated 22 October 2026.

On this page

  1. What this checker shows
  2. Search crawlers, AI search crawlers and training crawlers
  3. How to fix a block you didn't mean
  4. Where robots.txt stops

More free tools

  • Free SEO checker
  • Snippet preview
  • Business schema generator
  • llms.txt generator
  • Keyword overview
  • Website authority checker
  • Backlink checker
  • AI brand visibility checker
  • Website traffic checker
  • Competitor analysis
  • Free local audit

More free tools

Free toolFree SEO checker40+ checks on your home page in 8 areas, in one report: score, what is critical, warnings, speed on phone and computer, links, social previews, local details and accessibility.Free toolSnippet previewSee your title and description the way Google shows them on desktop and phone, with a pixel-width check and plain warnings.Free toolBusiness schema generatorFill in your business details and get LocalBusiness or Organization JSON-LD, checked and ready to paste into your site.Free toolllms.txt generatorWrite an llms.txt for your site in a form: a summary and your key pages, in the proposed format, ready to download.Free toolKeyword overviewHow often a search is typed into Google, how hard page one is, what people want, who holds the top 10, and related searches. Three a day, free.Free toolWebsite authority checkerA 0 to 100 Authority Score for any website, from the links pointing to it, with the numbers behind it.Free toolBacklink checkerHow many websites link to a site, and its ten strongest links with the words used and the linking site's strength.Free toolAI brand visibility checkerAsk ChatGPT, Gemini and Google's AI one question your customers ask, and see whether each names your business or cites your site.Free toolWebsite traffic checkerAn estimate of any website's monthly visits from Google, its last 12 months, and how it compares with its closest competitors.Free toolCompetitor analysisThe five websites that show up on Google for the most of the same searches as you, with shared searches, size and strength.Free toolFree local auditType your business name. See how complete your Google profile is, your rating against the top 3 nearby, where you show on the map, your details on five directories, and what to fix first.

Read next

Termrobots.txtA text file at a site's root that tells crawlers which parts of the site they may or may not crawl.GuideHow to get your business mentioned in ChatGPT answersChatGPT names businesses it can read and trust. Let its crawler in, describe your business clearly everywhere, and earn mentions on sites it already reads.Termllms.txtA proposed markdown file at a site's root that summarises the site and links to its key pages for AI tools.

Want this checked every day, and fixed?

ParSEO runs these checks and many more on every page, asks AI assistants what they say about you, and fixes what holds you back with your OK.

Start your Day 0
ParSEO

Own every search. Google and AI answers, fixed with your OK and proven with dates.

Ask us on WhatsApp

Product

  • How it works
  • Pricing
  • Free tools
  • Integrations
  • Results
  • Help
  • Security
  • Changelog

Who it's for

  • Clinics
  • Restaurants and cafes
  • CA and tax firms
  • Home services
  • All industries

Learn

  • Blog
  • SEO, AEO and GEO terms
  • SEO vs AEO vs GEO
  • Compare tools
  • Alternatives

Company

  • About
  • Contact
  • Privacy
  • Terms
© 2026 ParSEO. Made in India by Welvom AI, makers of CONRA AI.