Known Agents – Track AI Bots and Crawlers, Block Scrapers, Analyze LLM Referral Traffic vs AI Agent Analytics by Salespeak

Track crawlers, scrapers, LLM assistants, and AI agents on your website. Generate a robots.txt that blocks AI bots. Formerly Dark Visitors. Track, control, and optimize your website for AI agents and bots You have 40% more visitors than you think. Protect your content from unwanted AI scraping, and turn the rising wave of AI agents, LLM assistants, and other bots visiting your website into a new growth channel for your business. For full details, visit the Known Agents website. 1. AI Agent & Bot Analytics Get realtime visibility into the hidden ecosystem of crawlers, scrapers, and AI agents browsing your website Watch traffic from crawlers and scrapers as it happens: Track the hidden activity of all known artificial agents visiting your website in real time. Replay AI agent navigation patterns: Follow a single agent’s session as it navigates your website. Find where agents get stuck, identify dead ends, and help them complete their tasks. Catch bots spoofing legitimate agents: Identify bad actors and unauthorized scrapers pretending to be legitimate agents like Googlebot or Bingbot. Never miss a bot traffic spike: Stay informed when an agent’s traffic starts surging or overloading your server. Reveal which agents visit most, and which pages they target: Break down bot traffic by page to understand what they’re interested in. Export reports to CSV: Download agent activity reports for custom analysis. 2. LLM Citation & Referral Tracking (GEO, AEO) Measure citations and human click-throughs from AI chat platforms like ChatGPT, Claude, and Gemini Increase citations in AI chat responses: See which pages LLMs recommend most often. Double down on content that’s working. Find out which AI platforms are sending you traffic: Measure how many humans are clicking through to your website from a citation. Export reports to CSV: Download LLM referral reports for custom analysis. 3. Automatic Robots.txt & Bad Bot Detection Protect your content and reduce server cost by serving a robots.txt that updates continuously Set robots.txt rules for entire categories of bots: Control access for entire agent categories including AI crawlers, scrapers, SEO bots, and more. One selection covers every current and future agent in that group. We track new bots, you don’t have to: Your robots.txt is updated as new agents emerge, so you never have to manage them yourself or make manual edits. Block misbehaving crawlers and scrapers: Detect and block bots that ignore your robots.txt rules using the WordPress plugin, or our automation detectors. Demos Demos can be found on the home page. Further Details This plugin connects your website to Known Agents. It uses the Known Agents API to connect to your account and provide functionality. By signing up, you agree to the Terms of Service and Privacy Policy. Are you finding this plugin useful? Please leave us a ★★★★★ review! Questions or issues? Ask us for help.

AI agents are visiting your website hundreds of times a month. ChatGPT, Claude, Perplexity, Gemini. They are reading your pages, evaluating your content, and deciding whether to recommend you to their users. Can you see any of it? Now you can. See it in action: Based on analysis of 640,000+ AI agent crawls across B2B websites, AI agent traffic is one of the fastest-growing — and most invisible — sources of website activity. This plugin gives you complete visibility, plus the tools to act on what you find. AI Search Analytics Monitor how often AI tools access and cite your content over time — like Google Analytics, but for AI agent traffic. See trends by day, week, and month across all major AI platforms. Agent Identification & Analysis Identify exactly which AI agents are scanning your site: ChatGPT, Claude, Perplexity, Gemini, and more. Understand their behavior patterns and which content draws the most visits. Page Performance Intelligence See exactly which pages ChatGPT, Claude, and other AI agents actually read when answering questions. This is the foundation of effective Answer Engine Optimization (AEO). AEO Content Gap Detection Discover what critical information is missing from your most-accessed pages. Know what to add so AI agents can confidently recommend your site when users ask relevant questions. What Makes This Different: The Optimization Layer Most AI analytics plugins stop at tracking. This one also lets you optimize what AI agents see. Serve LLM-optimized versions of your pages — restructured with structured Q&A, comparison data, and testimonial cues — so ChatGPT and Claude get your complete story, not just whatever text they happen to parse. Analytics + action, not just a dashboard. This is the only WordPress plugin that tracks AI agent visits AND optimizes what they see. Cache Compatibility This plugin includes built-in support for bypassing cache on AI bot requests for: * WP Engine – via wpe_is_cacheable filter * WP Rocket – via do_rocket_generate_caching_files filter * LiteSpeed Cache – via litespeed_can_cache filter and LSCACHE_NO_CACHE constant * W3 Total Cache – via w3tc_can_cache filter * WP Super Cache – via wp_super_cache_late_init filter * SiteGround Optimizer – via sg_optimizer_cache_is_enabled filter * Kinsta – via cache-control headers * Cloudflare APO – via cookie-based bypass Server-Level Caching (Nginx, Varnish, CDN) If you use server-level caching that runs BEFORE PHP/WordPress, you may need additional configuration: For Nginx FastCGI Cache, add to your server config: if ($http_user_agent ~* "(ChatGPT|GPTBot|ClaudeBot|PerplexityBot|Claude-User|Claude-Web|Google-Extended|bingpreview|OpenAI)") { set $skip_cache 1; } For Varnish, add to your VCL: if (req.http.User-Agent ~ "(?i)(ChatGPT|GPTBot|ClaudeBot|PerplexityBot|Claude-User|Claude-Web|Google-Extended|bingpreview|OpenAI)") { return (pass); } For Cloudflare (non-APO), create a Cache Rule: – When: User Agent contains "ChatGPT" or "GPTBot" or "ClaudeBot" or "PerplexityBot" – Then: Bypass cache The plugin sends X-Salespeak-AI-Bot: 1 header for AI bot requests to help with debugging. Safety & Performance This plugin is designed to have ZERO impact on normal visitors: * All cache bypass logic only executes when the User-Agent matches a known AI bot pattern * For non-AI requests, all filter callbacks immediately return the original value (pass-through) * No headers, cookies, or constants are set for normal visitor requests * AI bot detection is cached per-request to avoid repeated regex checks * The plugin only modifies behavior for an explicit list of known AI bot User-Agent strings: * ChatGPT-User, GPTBot, Claude-User, Claude-Web, ClaudeBot * PerplexityBot, Google-Extended, bingpreview, OpenAI * Applebot-Extended, CCBot, anthropic-ai, cohere-ai External Services This plugin integrates with the following external services: Salespeak Analytics API Service: Salespeak’s event tracking and analytics service Purpose: Logs AI agent visits and user interactions for analytics and optimization purposes Data Sent: When AI agents are detected, the plugin sends: – Website URL being visited – Agent type (ChatGPT, Claude, GPTBot, etc.) – Organization ID (for account association) – Request ID (for tracking) – Campaign ID (default value for organization tracking) When Data is Sent: Only when AI agents are detected visiting your website Service Provider: Salespeak (https://salespeak.ai) Terms of Service: https://salespeak.ai/terms Privacy Policy: https://salespeak.ai/privacy Salespeak Content Delivery Network (CDN) Service: Amazon S3-based content delivery for serving AI-optimized content Purpose: Serves alternate, LLM-optimized content to detected AI agents Data Sent: No data is sent to this service — it only serves content based on URL requests When Data is Sent: Not applicable — this is a content delivery service only Service Provider: Amazon Web Services (AWS) S3, managed by Salespeak Terms of Service: https://aws.amazon.com/service-terms/ Privacy Policy: https://aws.amazon.com/privacy/ Privacy and Data Handling This plugin only sends data when AI agents are detected visiting your website. No personal user data is collected or transmitted. All data is used solely for analytics and content optimization purposes. The plugin respects user privacy and does not track individual human visitors.