AJ Agent Crawl Optimizer – AI Readiness Scanner & llms.txt for WordPress vs WebExpertDev AI Customer Support Assistant

AJ Agent Crawl Optimizer is a free WordPress plugin that scans your site against 21 AI-agent-readiness checks — the same set used by Cloudflare’s isitagentready.com — grades it on a Level 0–5 readiness ladder, fixes failing checks in one click, and lets you hand-curate the llms.txt that AI agents like ChatGPT, Claude, and Perplexity read. It makes your site legible to AI agents — and, since 2.0, proves it. The built-in readiness scanner runs 21 checks against your live site, grades you on the Level 0–5 agent-readiness ladder (Not Ready → Basic Web Presence → Bot-Aware → Agent-Readable → Agent-Integrated → Agent-Native), and shows the full evidence trail — every request and response — behind every verdict. Failing checks get a Fix now button that enables the right feature and re-scans that single check to prove it went green; anything the plugin can’t fix in WordPress (DNS records, server config) gets a copy-paste prompt for your coding agent. This is the technical, agent-readiness layer of answer engine optimization (AEO) and generative engine optimization (GEO): it improves your AI visibility by making your content legible to AI agents through llms.txt, JSON-LD, Markdown negotiation, and AI crawler rules. It is not a content SEO plugin — it doesn’t write titles or meta descriptions, and it doesn’t track brand mentions or AI citations. It prepares your WordPress site so AI search engines and agents can read, trust, and route on it, then verifies that over real HTTP. The scan → fix → verify loop (new in 2.0) Dashboard (Agent Ready → Dashboard) — segmented category gauge, Level badge, and a “Next level” panel listing exactly which checks unlock the next level. 21 checks across 5 categories — Discoverability (robots.txt, sitemap, Link headers, DNS-AID), Content Accessibility (Markdown negotiation), Bot Access Control (AI bot rules, Content Signals, Web Bot Auth), API/Auth/MCP Discovery (API catalog, OAuth discovery, OAuth Protected Resource, auth.md, MCP server card, A2A agent card, Agent Skills, WebMCP), and Commerce (x402, MPP, UCP, ACP, AP2 — informational, never scored). Evidence timelines — fetch/parse/conclude steps with request/response snapshots, so you can audit why a check passed or failed (and catch page caches or server rules silently breaking your endpoints). One-click fixes with verification — 9 checks are fixable in one click; the plugin re-scans just that check afterwards. A bulk “Fix all safe items” sheet handles the rest. Hosting diagnosis — the plugin writes no files (every endpoint is served by WordPress), but some hosts block the requests before WordPress runs: nginx dot-path rules 403 /.well-known/*, and static-file rules 404 /llms.txt. When a feature is enabled and the server blocks it, the Dashboard says so and hands you copy-paste nginx and Apache fixes. REST API — POST /wp-json/ajaco/v1/scan, POST /wp-json/ajaco/v1/scan/check, POST /wp-json/ajaco/v1/fix, POST /wp-json/ajaco/v1/llms/preview, and a public GET /wp-json/ajaco/v1/health. WP-CLI — wp agent-ready scan --format=summary|json|agent, wp agent-ready fix |--all-safe, wp agent-ready status for agency fleets and AI agents operating over SSH. Each publishing capability remains a separate toggle under Agent Ready → Settings and ships opt-in (everything starts off). On first activation, a Quick Setup wizard suggests sensible defaults based on your environment (for example, it skips JSON-LD when an SEO plugin is detected so you don’t get duplicate structured data) — re-runnable any time. Curated llms.txt — you decide what agents read Most plugins generate llms.txt from fixed rules and hand you the result. This one hands you the controls. A dedicated editor (Agent Ready → llms.txt) — write the intro that tells an agent what your site is for, choose which content types appear (custom post types and WooCommerce products are detected automatically), and set each section’s heading, item count and order. Add your own Markdown block above or below. Live preview — see exactly what /llms.txt will serve, with entry, byte and token counts, before you save. Per-post control — every post and page gets an “Agent Ready (llms.txt)” panel in the editor: keep an entry out of the agent indexes entirely, or write a Summary for AI agents that overrides the excerpt. An excerpt is written for a human skimming; that line is written for a model deciding whether to fetch the page — and it is the single highest-leverage field in the whole plugin. /llms-full.txt serves the full content of the same curated entries. Password-protected content is always excluded from both. The defaults keep the same sections and ordering as the previous automatic output (2.0 adds a one-line pointer to /llms-full.txt), so upgrading needs no attention until you decide to curate. Discovery — help agents find what your site offers API Catalog (RFC 9727) — /.well-known/api-catalog linkset advertising your REST API, plus a Link: rel="api-catalog" header on every response so agents discover it from any URL. MCP Server Card (SEP-1649 draft) — /.well-known/mcp/server-card.json describing the site to MCP-aware agents. Agent Skills Index (RFC v0.2.0) — /.well-known/agent-skills/index.json listing six skills (content-query, posts-read, pages-read, media-library, categories, tags) plus per-skill SKILL.md artifacts with verifiable sha256 digests. llms.txt + llms-full.txt (per llmstxt.org) — /llms.txt is a curated LLM-readable index: you write the intro, choose which post types appear (custom post types and WooCommerce products included), set each section’s heading, count and order, and add your own Markdown block. /llms-full.txt serves the full content of those same entries as Markdown. Password-protected content is always excluded. Per-post curation — an “Agent Ready (llms.txt)” panel in the editor (block editor and classic) lets an author keep a page out of the agent indexes entirely, or write a Summary for AI agents that overrides the excerpt — a line aimed at a model deciding whether to fetch the page, not a human skimming. IndexNow — non-blocking ping to Bing and Yandex on every post publish so search engines re-crawl within minutes. Presentation — format content for agents Markdown Negotiation — when a request includes Accept: text/markdown, the page is served as clean Markdown with X-Markdown-Tokens for context budgeting. Browsers (which send text/html) are completely unaffected. JSON-LD Schema — Schema.org structured data: WebSite, Organization, Article, BreadcrumbList, and auto-detected FAQPage. Logo resolved from your theme’s custom logo or site icon. OpenAPI 3.0.3 — /openapi.json (and legacy /?format=openapi) returns a complete spec generated dynamically from rest_get_server(), including plugin-registered REST routes. WebMCP Tools — registers four tools (search, posts, pages, site info) via navigator.modelContext.provideContext() for browsers that support the W3C WebMCP draft. Declarations AI Bot Rules (new in 2.0) — to be cited by an AI engine, you must first allow its crawler. The plugin writes explicit robots.txt User-agent groups for the 15 AI crawlers readiness scanners check for (GPTBot and ChatGPT-User from OpenAI, Claude-Web, PerplexityBot, CCBot, Google-Extended, Bytespider, …), each allowed or blocked per a filterable per-bot policy. Groups are self-contained (they replicate core’s wp-admin protections) and honor “Discourage search engines”. Content-Signals — appends a Content-Signal: ai-train=no, search=yes, ai-input=no directive to robots.txt (inside an explicit User-agent: * group) declaring your AI-usage preferences per contentsignals.org. Composes with Yoast/Rank Math/AIOSEO — their additions are preserved, our directive lands last. auth.md (new in 2.0) — publishes /auth.md honestly documenting how agents authenticate to your REST API via Application Passwords: how a human creates one, Basic-auth usage, scope, and revocation. No fictional OAuth endpoints. How it compares to other AI / llms.txt plugins Most AI and llms.txt plugins only generate a file from fixed rules. AJ Agent Crawl Optimizer also scans your live site over real HTTP, scores it on the Level 0–5 ladder, shows the request/response evidence behind every verdict, and fixes failures in one click — then re-scans to prove the check turned green. It writes per-bot robots.txt rules for 15 AI crawlers and lets you hand-curate what AI agents read instead of auto-dumping every post. Scan → score → fix → verify, not generate-and-hope. Why use it No conflicts with your SEO plugin. JSON-LD auto-suppresses when Yoast SEO, Rank Math, All in One SEO, SEOPress, The SEO Framework, Slim SEO, Squirrly SEO, Schema Pro, or SASWP is active. Multisite-aware. Every endpoint also resolves at /{subsite}/... paths automatically. Cached. OpenAPI is cached for a day, llms.txt for an hour, with proper invalidation on plugin activation, theme switch, post changes, and setting toggles. Performance-conscious. Markdown handler runs at PHP_INT_MAX priority so it doesn’t break object-cache flushes or Query Monitor. IndexNow pings are non-blocking. Extensible. Seventeen filter hooks let you customize skills, schemas, bot policies, llms.txt curation, scan behavior, and content. See “For Developers” below. Accessible. The Dashboard gauge carries a dynamic aria-label, copy buttons announce success via aria-live, and decorative icons are hidden from screen readers. For developers The plugin exposes seventeen filter hooks for extension. Examples: add_filter( 'ajaco_required_capability', function () { return 'edit_posts'; } ); Delegate plugin access to a non-admin role. add_filter( 'ajaco_skill_definitions', function ( $skills ) { return $skills + [ 'products' => [ 'type' => 'information-retrieval', 'description' => 'WooCommerce products', 'endpoint' => rest_url( 'wc/v3/products' ) ] ]; } ); Register custom skills that ship in the Agent Skills Index and are served as SKILL.md artifacts with verifiable sha256 digests. add_filter( 'ajaco_content_signal', function () { return 'ai-train=yes, search=yes, ai-input=yes'; } ); Customize the Content-Signal directive (e.g. permit AI training). Other hooks: ajaco_api_catalog_linkset, ajaco_mcp_server_card, ajaco_json_ld_graph, ajaco_openapi_spec, ajaco_llms_txt_content, ajaco_llms_full_txt_content, ajaco_llms_post_types, ajaco_llms_exclude_post, ajaco_auth_md_content, ajaco_ai_bot_list, ajaco_ai_bot_policy, ajaco_commerce_signals, ajaco_scan_sslverify, ajaco_active_seo_plugin. The Help tab (top right of the settings screen) lists all of them with descriptions. Per-post curation is stored in two post meta fields you can read or set programmatically: _ajaco_llms_exclude (boolean) and _ajaco_llms_summary (string, 300 chars). Both are registered with show_in_rest, so they round-trip through the REST API and WP-CLI (wp post meta update 42 _ajaco_llms_summary "..."). External services This plugin connects to external services only in two explicitly user-triggered situations: the opt-in IndexNow feature, and DNS lookups made while an administrator runs a readiness scan. DNS-over-HTTPS resolvers (cloudflare-dns.com, dns.google) — during scans only What they are and what they’re used for: the readiness scanner’s DNS-AID check queries public DNS-over-HTTPS resolvers (Cloudflare’s https://cloudflare-dns.com/dns-query, with Google’s https://dns.google/resolve as fallback) to look up _agents SVCB/HTTPS/TXT records for your own domain. When data is sent: only while a scan that an administrator started (Dashboard, REST, or WP-CLI) is running. The scanner also sends ordinary HTTP requests to your own site’s URLs to verify your endpoints. What data is sent: only DNS query names derived from your own domain (e.g. _index._agents.example.com). No visitor information, IP addresses of visitors, or content is sent. Cloudflare’s and Google’s resolver privacy policies apply: Cloudflare 1.1.1.1 privacy: https://developers.cloudflare.com/1.1.1.1/privacy/public-dns-resolver/ Google Public DNS privacy: https://developers.google.com/speed/public-dns/privacy IndexNow (api.indexnow.org) What it is and what it’s used for: IndexNow is an open protocol (originally from Microsoft Bing and Yandex) that lets sites notify search engines the moment a URL is published or updated, so search engines can re-crawl within minutes instead of days. When data is sent: only when the IndexNow feature toggle is turned on AND an IndexNow API Key is configured on the settings page. In that case, every time a post of a public post type transitions to the publish status, the plugin fires a single non-blocking HTTPS POST to https://api.indexnow.org/indexnow. What data is sent: the request body is a JSON document containing exactly three fields — your site’s host (e.g. example.com), your IndexNow API key (which you generated yourself at Bing Webmaster Tools), and the permalink URL of the post that was just published. No visitor information, IP addresses, user-agents, or post content is sent. If the IndexNow feature toggle is off (the default) and no scan is running, the plugin makes no outbound network requests of any kind. This service is provided by Microsoft (Bing) and the IndexNow project. Their terms and privacy policies apply: IndexNow protocol documentation: https://www.indexnow.org/documentation IndexNow FAQ and terms: https://www.indexnow.org/faq Microsoft Bing Webmaster Guidelines : https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a Microsoft Privacy Statement: https://www.microsoft.com/en-us/privacy/privacystatement Privacy AJ Agent Crawl Optimizer stores data only on your own server — there is no telemetry, no analytics, and no third-party logging. Specifically: Local data Plugin option rows in the WordPress options table store toggle states, the per-bot AI crawler policy, the most recent scan result (including its evidence snapshots of your own site’s responses), and the IndexNow API Key (stored as plain text — keep your database secure). Transients cache the OpenAPI document and the llms.txt / llms-full.txt bodies. These are deleted on plugin uninstall. No request data, IP addresses, User-Agents, or visitor information is recorded by the plugin. Outbound network calls The plugin makes outbound HTTP requests only in these cases: IndexNow — when the IndexNow toggle is on and a key is configured, the plugin sends a non-blocking POST to https://api.indexnow.org/indexnow on every post publish. The payload contains the site host, the IndexNow key, and the URL of the published post (no visitor data). Readiness scans — while an administrator-initiated scan is running, the scanner sends HTTP requests to your own site’s URLs (to verify endpoints exactly as an agent would see them) and DNS-over-HTTPS queries for your own domain to Cloudflare/Google resolvers (DNS-AID check). Scans never run automatically. Outside those two cases the plugin makes zero outbound network requests. All publishing features (manifests, JSON-LD, robots.txt) only respond to incoming HTTP requests; they never call out. Cookies The plugin does not set any cookies. Uninstall When the plugin is deleted (not just deactivated), all plugin options, the IndexNow key, and all transients are removed from the database. Multisite networks have every site cleaned in turn.

WebExpertDev AI Customer Support Assistant for stores using WooCommerce exposes lightweight, token-protected REST/OpenAPI tools that compatible AI support platforms can use for customer-safe product and order support. It is not a chatbot and does not include an AI model. It provides WooCommerce OpenAPI tools, setup assets, generated instructions, and admin configuration screens for stores that connect those tools to a compatible chat or AI assistant platform. Tawk.to compatibility: this plugin is Tawk.to Ready today and works with Tawk.to AI Assist compatible OpenAPI setup. It is also OpenAPI-compatible by design for AI assistants that support OpenAPI tool calling, HTTPS REST endpoints, and bearer authentication. Disclaimer: This plugin is not affiliated with, endorsed by, or sponsored by Tawk.to. All third-party trademarks belong to their owners. No external AI API calls, remote services, external trackers, account code, cart links, autoship actions, or write actions are included in the Free scope. Free Features Bearer-token REST security with hashed token storage. Public health and OpenAPI schema endpoints. Optional token requirement for product tools. WooCommerce product search with short-lived transient caching. Product availability by SKU or product ID. Verified order status requiring order number plus billing email or ZIP. Customer-safe status mapping for built-in and custom WooCommerce statuses. Customer-safe tracking summaries from supported local tracking metadata. Admin-controlled privacy-safe logs with enable/disable, clear logs, and retention cleanup. Tawk.to Ready setup prompt, knowledge block, test questions, and checklist. Generic OpenAPI-compatible setup guidance for compatible AI assistants. Optional Tawk widget embed using property ID and widget ID fields. Config export/import for non-secret plugin settings. AI behavior mode and industry preset scaffolding for prompt/knowledge generation. Free Scope Boundaries The Free plugin does not create carts, create reorder links, capture leads, process payments, modify orders, process refunds, send emails, send webhooks, or connect to external CRM, automation, shipping, or AI services. Developer extension points are present for compatible custom implementations, but those integrations are not included in the Free plugin. Optional Pro Add-On The Free plugin is fully usable on its own for safe product support, product availability, verified order status, tracking summaries, OpenAPI setup, and assistant-ready support configuration. For stores that want to turn support conversations into customer-click actions, WebExpertDev AI Customer Support Assistant Pro is an optional paid add-on hosted outside WordPress.org. Learn more about the optional Pro add-on on the WebExpertDev website. Pro adds separate paid workflows through separate Pro code: Lead Capture Secure Cart Links Secure Reorder Links Product Discovery AI Search Keywords Knowledge Builder Advanced Tracking support Pro secure links are signed, expiring, customer-click review links. They do not create orders, process payments, issue refunds, or secretly change carts. Free remains fully usable without Pro and does not include Freemius, payment, telemetry, account code, or Pro source. Third-Party Services This plugin does not call third-party APIs by itself and does not include a hosted AI service. If a site administrator enables the optional Tawk widget embed, visitor browsers may load the widget script from https://embed.tawk.to/ using the property ID and widget ID entered by the administrator. That widget service is provided by Tawk.to. Review Tawk.to’s privacy policy and terms before enabling the widget: Privacy policy: https://www.tawk.to/privacy-policy/ Terms of service: https://www.tawk.to/terms-of-service/ If a site administrator manually imports this plugin’s OpenAPI schema into Tawk.to AI Assist or another compatible OpenAPI-based AI platform, that platform may call this site’s REST endpoints using the bearer token the administrator provides. The plugin does not create a Tawk.to account, transmit API tokens to Tawk.to automatically, transmit tokens to any AI platform automatically, or imply any affiliation with Tawk.to. Security Model Protected endpoints use Authorization: Bearer TOKEN. Tokens are shown only immediately after generation and stored hashed. Order status is always protected and requires order number plus billing email or ZIP. The plugin does not return payment details, full addresses, internal notes, customer IDs, user IDs, private meta, fraud data, risk data, or backend routing. The optional Tawk widget embed uses safer ID-based fields for the Tawk property ID and widget ID. The plugin generates the widget URL from those IDs and does not accept arbitrary JavaScript embed code. Config exports exclude API tokens, token hashes, logs, order data, customer data, and secrets. Imports are sanitized and do not import secrets. Plugin settings are stored in the wcast_settings option. Privacy-safe logs are stored under wp-content/uploads/webexpertdev-ai-customer-support-assistant/logs/. Product search and generated prompt/knowledge caches use wcast_ transients when present. Uninstall cleanup is controlled from the plugin admin. By default, logs and generated caches are removed on uninstall, while settings are retained unless explicitly enabled for deletion. The plugin never deletes WooCommerce orders, products, customers, or order meta. Minimum Requirements PHP 7.4 or newer. WordPress 6.2 or newer. WooCommerce 7.1 or newer for live tools. If WooCommerce is inactive, the admin setup remains available and tools return unavailable responses. Platform Scope The Free plugin implements WooCommerce REST/OpenAPI tools under /wp-json/wc-ai-support/v1/. It is Tawk.to Ready today and OpenAPI-compatible by design for AI assistants that support OpenAPI tool calling and bearer authentication. Platform behavior varies, and this plugin does not claim official support for every AI platform.