This score measures whether AI answer engines can reach, read, parse and attribute this website — the prerequisite for ever being recommended by one.
Each band is scored independently, then weighted. A single blocked crawler can cost more than a dozen cosmetic issues.
Ordered by points actually lost, so a half-credit heavyweight outranks a fully-failed minor item. This list is the scope of work for an AI Readiness Sprint — nothing here is speculative, every line was measured today.
Question headings match the shape of a user prompt, which is how retrieval picks the passage it will quote.
These two strings are frequently the entire snippet a model sees before deciding whether the page is relevant.
Two live hostnames split every citation and backlink signal in half.
Models quote extractable spans. A page that opens with a hero slogan and no declarative answer gives the model nothing to lift.
IndexNow pushes new and changed URLs into Bing within minutes instead of weeks — and Bing's index is what ChatGPT Search reads.
Anonymous content is systematically discounted against attributed content.
Two independent tests per crawler: what robots.txt permits, and what the live server does when that crawler's user-agent actually knocks. The second test is the one that catches an edge or WAF quietly overriding the first.
| Crawler | What it feeds | robots.txt | Live edge response | Matched rule |
|---|---|---|---|---|
| ChatGPT-User OpenAI · critical |
ChatGPT live browsing (user asks about you) | Allowed | HTTP 200 | User-agent: * → Allow: / |
| Google-Extended Google · critical |
Gemini grounding + AI Overviews eligibility | Allowed | not probed | User-agent: * → Allow: / |
| Googlebot Google · critical |
Google index — feeds AI Overviews + Gemini grounding | Allowed | HTTP 200 | User-agent: * → Allow: / |
| OAI-SearchBot OpenAI · critical |
ChatGPT Search citations | Allowed | HTTP 200 | User-agent: * → Allow: / |
| Perplexity-User Perplexity · critical |
Perplexity live fetch on a user question | Allowed | HTTP 200 | User-agent: * → Allow: / |
| PerplexityBot Perplexity · critical |
Perplexity index + citations | Allowed | HTTP 200 | User-agent: * → Allow: / |
| bingbot Microsoft · critical |
Bing index — powers ChatGPT Search + Copilot | Allowed | HTTP 200 | User-agent: * → Allow: / |
| Claude-SearchBot Anthropic · high |
Claude search results | Allowed | HTTP 200 | User-agent: * → Allow: / |
| Claude-User Anthropic · high |
Claude live fetch on a user question | Allowed | HTTP 200 | User-agent: * → Allow: / |
| ClaudeBot Anthropic · high |
Claude model training corpus | Allowed | HTTP 200 | User-agent: * → Allow: / |
| GPTBot OpenAI · high |
GPT model training corpus | Allowed | rate-limited | User-agent: * → Allow: / |
| Amazonbot Amazon · medium |
Alexa answers | Allowed | HTTP 200 | User-agent: * → Allow: / |
| Applebot-Extended Apple · medium |
Apple Intelligence / Siri answers | Allowed | not probed | User-agent: * → Allow: / |
| CCBot Common Crawl · medium |
Common Crawl — the corpus almost every open model trains on | Allowed | HTTP 200 | User-agent: * → Allow: / |
| DuckAssistBot DuckDuckGo · medium |
DuckAssist answers | Allowed | HTTP 200 | User-agent: * → Allow: / |
| meta-externalagent Meta · medium |
Meta AI / Llama training | Allowed | rate-limited | User-agent: * → Allow: / |
| AI2Bot Allen Institute · low |
OLMo open training corpus | Allowed | not probed | User-agent: * → Allow: / |
| Bytespider ByteDance · low |
Doubao / TikTok AI | Allowed | not probed | User-agent: * → Allow: / |
| Diffbot Diffbot · low |
Knowledge-graph vendors | Allowed | not probed | User-agent: * → Allow: / |
| MistralAI-User Mistral · low |
Le Chat live fetch | Allowed | not probed | User-agent: * → Allow: / |
| PetalBot Huawei · low |
Petal AI search | Allowed | not probed | User-agent: * → Allow: / |
| Timpibot Timpi · low |
Timpi index | Allowed | not probed | User-agent: * → Allow: / |
| YouBot You.com · low |
You.com answers | Allowed | not probed | User-agent: * → Allow: / |
| cohere-ai Cohere · low |
Cohere training | Allowed | not probed | User-agent: * → Allow: / |
Expand a band to see all checks in it.
| Cloudflare AI controls are not withholding the site from AI engines | N/A | Cloudflare in front, no managed AI block detected |
| AI crawlers are allowed in robots.txt | Clear | all 24 tracked AI crawlers allowed |
| AI crawler user-agents are served by the edge/WAF | Clear | all 12 probed crawlers received HTTP 200 · 2 rate-limited, excluded from scoring |
| Site responds to an anonymous fetch | Clear | HTTP 200 in 0.11s from https://byteflowtech.in |
| Canonical origin is HTTPS with a valid certificate | Clear | canonical origin resolved to https://byteflowtech.in |
| robots.txt is served correctly | Clear | 13 lines, 2 User-agent groups |
| No opt-out AI directives set | Clear | no noai / noimageai / noml directives |
| robots.txt parses without errors | Clear | no syntax errors |
| Answer-first block near the top of the page | Partial | 28-word lead paragraph: "Automation. Engineered for Enterprises. An IT and software development company in Ujjain and Indore, ByteFlow …" |
| Sub-headings phrased as real questions | Blocked | 0 of 11 H2s are question-shaped |
| Title and meta description are specific and sized right | Blocked | title 76 chars: "IT & Software Development Company in Ujjain & Indore | ByteFlow Techno" · description 262 chars |
| Healthy text-to-markup ratio | Partial | 4.2% of the HTML payload is extractable text (8,888 chars text / 211,709 chars HTML, 113,477 chars inline JS) |
| Content is present in the server HTML (no JS required) | Clear | 1294 words of extractable text in the raw HTML |
| Homepage is indexable | Clear | meta robots: index, follow |
| llms.txt published | Clear | /llms.txt served, 2099 bytes |
| Exactly one H1 and a clean heading hierarchy | Clear | 1 H1, 11 H2, 30 headings total |
| Structured lists or tables for facts | Clear | 9 lists, 0 tables |
| Enough substance on the page to answer with | Clear | 1294 words of main content on the homepage |
| Semantic HTML landmarks present | Clear | found: main, section, nav, footer, header |
| Canonical URL declared | Clear | https://byteflowtech.in/ |
| Page language declared | Clear | <html lang="en"> |
| Images carry descriptive alt text | Clear | 2 of 2 images have alt text |
| Article markup carries author and dates | N/A | 0 Article nodes · 0 with author · 0 with a date |
| JSON-LD structured data present | Clear | 6 of 6 sampled pages carry JSON-LD; types found: Answer, BreadcrumbList, City, ContactPoint, Country, FAQPage, GeoCoordinates, ListItem, OpeningHoursSpecification, Organization |
| Organization / LocalBusiness entity declared | Clear | 9 Organization/LocalBusiness node(s) |
| sameAs links disambiguate the brand | Clear | 18 sameAs link(s): https://www.facebook.com/byteflowtech, https://www.instagram.com/byteflowtech.i, https://maps.app.goo.gl/8UogLvHHcLe21ftm |
| FAQPage markup with real Q&A pairs | Clear | 4 FAQPage node(s), 27 question(s) |
| All JSON-LD blocks parse | Clear | 31 blocks parsed cleanly |
| The offering itself is marked up | Clear | found: Service |
| BreadcrumbList declared | Clear | BreadcrumbList present |
| WebSite node declared | Clear | WebSite present |
| www / non-www resolve to one canonical host | Blocked | https://www.byteflowtech.in/ → HTTP 200 |
| IndexNow key file in place | Blocked | no IndexNow key file detected in the sitemap |
| RSS/Atom feed available | Blocked | no feed at the conventional paths |
| XML sitemap published and parseable | Clear | 113 URLs from https://byteflowtech.in/sitemap.xml |
| Server responds fast enough for a crawler | Clear | homepage answered in 0.11s |
| Sitemap declared in robots.txt | Clear | declared: https://byteflowtech.in/sitemap.xml |
| HTML payload is a reasonable size | Clear | 207 KB of HTML on the homepage |
| Missing URLs return a real 404 | Clear | probe URL returned HTTP 404 |
| Name, address and phone stated in plain text | Partial | phone ✓ · email ✓ · PIN code ✗ |
| Content carries a named author | Blocked | no visible bylines |
| Outbound citations to authoritative sources | Blocked | 0 authoritative outbound domain(s): none |
| A real About page exists | Clear | https://byteflowtech.in/about/ — 327 words |
| A Contact page exists | Clear | https://byteflowtech.in/contact/?intent=transformation |
| Content shows a recent date | Clear | most recent year mentioned: 2028 |
| Official profiles linked from the site | Clear | 4 profile link(s): wa.me, www.facebook.com, www.instagram.com, www.linkedin.com |