Crawler access
robots.txt per AI bot (GPTBot, ClaudeBot, PerplexityBot…), Cloudflare anti-bot challenges, redirects, TTFB, noindex.
Open-source · 32 real checks · free
Is your site ready for AI?
Most AI crawlers don't run JavaScript. One header can hide you from ChatGPT. We audit what GPTBot, ClaudeBot and PerplexityBot actually see — and tell you exactly how to fix it.
or run it in your terminal
32
real checks
5
score dimensions
21
AI crawlers tracked
~5s
median scan time
CLI · open source · no install
Audit from your terminal
The full scanner ships as an open-source CLI. One command, no install, no signup — and exit codes that gate your CI when the score drops.
- --json for your custom services
- --llm for an AI-agent fix plan
- exit 1 below threshold (CI gate)
isready — terminal
$ npx isreadyai yourdomain.comGitHub Action · README badge · CI gate
Ship AI-ready, every deploy
Add one step to your workflow: the full report lands in the job summary, the run fails when your score drops below the threshold, and the live badge shows the world you're AI-ready.
- Score gate — exit 1 below your threshold
- Full Markdown report in the job summary
- score and grade outputs for downstream steps
- Free for open-source projects
isreadyai — AI readiness audit
isreadyai/isreadyai
Use in your workflow
- name: AI readiness audit
uses: isreadyai/isreadyai@v1
with:
url: ${{ env.DEPLOY_URL }}
threshold: 80Live README badge
Like a coverage badge — always your domain's current score, re-scanned automatically.
How it works
Grounded in how AI crawlers actually behave — not SEO folklore.
Fetch like an AI crawler
We request your pages exactly like GPTBot and ClaudeBot do: raw HTTP, no JavaScript, short timeouts, honest user-agent.
Read what they read
We parse the raw HTML exactly as GPTBot and ClaudeBot do — no JavaScript, because they don't run it either. Empty app shells and content that only appears after hydration get flagged.
Score and fix
32 evidence-based checks across 5 dimensions produce a 0–100 score, and every finding ships with a concrete fix, impact and effort.
What we check
Every check is sourced from official crawler docs and peer-reviewed GEO research.
Rendering
Empty app shells, missing semantic landmarks, scripts with no server-rendered text, noscript fallbacks.
Structured data
JSON-LD presence and types, meta basics, Open Graph, author & E-E-A-T signals, lang attributes.
Trust & security
HTTPS enforcement, TLS validity, HSTS, mixed content.
Content (GEO)
Content depth, heading structure, statistics & citations — the signals that raise visibility in generative answers.
Honest signals
llms.txt and Content Signals, reported honestly as informational: server logs show AI crawlers fetch llms.txt in ~0.1% of visits — we never score it.
Why AI readiness matters now
AI assistants are becoming a primary discovery channel. Cloudflare measured AI crawlers at roughly 50 billion requests per day in 2025 — nearly 1% of all web traffic — and GPTBot traffic grew over 300% year-over-year. Yet the rules are different from classic SEO: GPTBot, ClaudeBot, PerplexityBot and OAI-SearchBot read raw HTML only and never execute JavaScript, while Googlebot renders everything. A React or Vue app with client-side rendering can rank on page one of Google and still be a blank page to every AI assistant.
The research is equally clear about content. The peer-reviewed GEO study (Aggarwal et al., KDD 2024) tested 10,000 queries across multiple generative engines and found that adding quotations improved visibility in AI answers by 41%, statistics by 32%, and citing credible sources by up to 115% for lower-ranked sites. As the authors put it:
“Generative engines synthesize answers from a handful of sources — the competition is no longer for a ranking position, but for being part of the answer itself.”— GEO: Generative Engine Optimization, KDD 2024
Meanwhile, one misconfigured header can erase you: Cloudflare's AI-bot blocking now protects millions of sites, and many owners don't realize it also removes them from ChatGPT and Perplexity answers. Independent server-log studies show 2.5 million+ websites already disallow AI training — a legitimate choice, but one that should be made deliberately, not by accident. That is exactly what this scanner makes visible: 32 evidence-based checks, each with the observed value, the consequence, and the precise fix.
Primary sources: GEO research paper (arXiv) · OpenAI crawler documentation · Anthropic crawler documentation · Cloudflare AI Crawl Control
Frequently asked questions
›Do AI crawlers really not execute JavaScript?
Correct. GPTBot, ClaudeBot, PerplexityBot and OAI-SearchBot read raw HTML only — they do not render. Only Googlebot and Bingbot run JavaScript. If your content appears only after hydration, most AI systems see an empty page.
›What does the score mean?
It's a weighted 0–100 across five dimensions: crawler access (25%), structured data (30%), rendering (20%), content/GEO (15%), trust (10%). Informational signals (llms.txt, Content Signals) are reported but never scored. Scoring is versioned and open source.
›Is it really free?
Yes — the scanner is MIT-licensed open source, the web scan is free, and you can run it yourself with npx isreadyai. Premium adds AI-generated fix plans, monitoring and history.
›Can you scan apps behind a login?
Public pages only for now. GitHub-connected source scanning for login-gated SaaS is on the roadmap.
›Should I add llms.txt?
It can't hurt, but don't expect miracles: no major AI provider has confirmed consuming it, and Google has said it doesn't use it. We report it as informational and never let it affect your score.
›Can I run this in CI or from the terminal?
Yes. Run npx isreadyai yourdomain.com for the full audit in your terminal, with --json for machines and --llm for a fix plan you can paste into an AI coding agent. The GitHub Action gates deployments on a minimum score and writes the full report to the job summary — free for open-source projects.