AI Search Readiness (ai-search-readiness)

Summary

llms.txt, AI crawler rules in robots.txt, structured data depth, and snippet-oriented content signals.

Classification

• Category: SEO
• Plugin id: ai-search-readiness
• Version: 1.0.0
• Target types: WEBSITE, WEBSHOP
• Weight class: LIGHT
• Risk level: not set on plugin object
• Browser required: No (HTTP / Node / other — see source)
• Tier / group: Tier 1 — HTTP-only

Schedule

• scheduleHints: { defaultInterval: '0 4 * * 1', priority: 3 }
• Prerequisites: None declared on this plugin.

Configurable inputs

Public keys are derived from inputSchema: z.object({ ... }) (underscore-prefixed keys are runtime-only and omitted here).

Field	Notes
timeout	Zod field in inputSchema — see source for defaults, min/max, and .describe().

Runtime / injected config

• None documented beyond standard worker context (target, config, logger, reportProgress). Underscore-prefixed keys are internal.

Checks that produce findings

These checks run inside this plugin during a single test run. They are not separate tests in the database — each becomes a Finding linked to the Run for ai-search-readiness.

AI crawlers checked in robots.txt: GPTBot, Google-Extended, ClaudeBot, anthropic-ai, CCBot, Bytespider, Applebot-Extended, PerplexityBot.

Check	Typical severity	Why
No /llms.txt	INFO	AI crawlers use llmstxt.org files to discover site purpose and key URLs; absence reduces control over how models summarize you.
llms.txt formatting issues	INFO	File exists but lacks # title or markdown [text](url) links — weaker machine-readable site map for AI systems.
Inconsistent AI crawler policy (some blocked, some allowed)	WARNING	Mixed robots rules mean ChatGPT might index you while Claude cannot — uneven brand presence across AI search products.
All AI crawlers blocked	INFO	Intentional opt-out from AI indexing; content will not appear in ChatGPT/Gemini/Perplexity-style results.
Missing expected schema.org type for site type	WARNING for Product on WEBSHOP; INFO for others	JSON-LD types expected for target.type (Organization, Product, BreadcrumbList, Article, WebSite) improve entity understanding and rich results.
No FAQPage structured data	INFO	FAQ schema helps AI and Google extract Q&A pairs for direct answers.
JSON-LD validation errors	WARNING	Parse errors or missing Product/FAQPage required fields prevent search engines from using structured data.
No sameAs knowledge graph links	INFO	sameAs ties the brand to Wikipedia/social profiles — important for entity disambiguation in AI answers.
Heading hierarchy issues	INFO	Skipped levels (e.g. H2→H4) make content structure harder for crawlers to segment.
Weak introductory content (< 50 chars)	INFO	First paragraph under 50 characters is a poor snippet source for AI-generated summaries.
Meta description wrong length for AI snippets	INFO	Meta description present but outside 120–160 characters — suboptimal for snippet extraction (only when description exists).

How each check works

1. llms.txt — GET /llms.txt; non-empty 200 body required. Format check: must start with # and contain markdown links […](…). HEAD /llms-full.txt is recorded in metrics only (no finding).
2. robots.txt — GET /robots.txt, parse with shared parseRobotsTxt / isAgentBlocked. Inconsistent = some AI agents blocked, some allowed. All blocked = every agent in the list blocked.
3. Homepage parse — fetchAndParse target URL; extract all application/ld+json blocks, validate Product (name, offers/price, image) and FAQPage (mainEntity non-empty).
4. Schema expectations — Per SCHEMA_EXPECTATIONS and target.type: Organization/LocalBusiness/Corporation, Product (WEBSHOP), BreadcrumbList (WEBSHOP), Article/BlogPosting (WEBSITE), WebSite — each missing type emits one finding with configured severity.
5. sameAs — Search parsed JSON-LD details or raw script text for "sameAs".
6. Content signals — Heading scan for level skips; first paragraph from main p, article p, .content p, body p; meta description length 120–160 (passes if description empty).
7. Scoring — Pass/fail tally drives score (passed/total × 100); status from statusFromFindings.

Caveats and concerns

• Homepage-only for schema/content — robots and llms.txt are origin-level; JSON-LD and copy checks use one URL.
• Static HTML — Client-rendered JSON-LD or headings are invisible.
• FAQ always suggested — INFO finding even when the site has no FAQ content strategy.
• Blocking may be intentional — “All crawlers blocked” is INFO, not a failure.
• Product severity — Missing Product schema on WEBSHOP is WARNING; other missing types are INFO.

Source

packages/test-plugins/src/plugins/ai-search-readiness.ts