AI Crawler and SEO Manager by akbansa vs Prime SEO – AI SEO, LLMs.txt Generator, AI Bots Manager & AI Crawler Stats

The AI Crawler and SEO Manager by akbansa plugin generates an llms.txt file for crawler discovery and logs visits by AI-based crawlers such as GPTBot, ClaudeBot, and others. It also provides an analytics page in the WordPress admin dashboard to view crawler activity by path, URL, and timestamp. Key Features: Generates llms.txt file for crawler discovery. Removes llms.txt on plugin uninstall Schedules automatic generation of llms.txt daily at midnight (frequency configurable: daily, twice daily, hourly) Uses atomic file writes for llms.txt and sitemap.xml for safety Integrates with Yoast SEO, Rank Math, or All in One SEO for metadata (falls back to DB if not present) Admin panel to select/exclude categories and content types for llms.txt Logs every AI crawler request from the frontend (user agent, crawl info, page URL, IP, referrer) Filters crawlers using config.json and stores all details in a dedicated DB table Analytics dashboard in admin to view recent crawler activity Robust AJAX security and error handling Detects and logs visits from AI crawlers based on their User-Agent. Stores crawler logs in a database for analysis. Supports popular AI-based crawlers (e.g., OpenAI, Anthropic, GPTBot, etc.). Option to customize the list of crawlers via a config.json file. Short Description Advanced AI-powered SEO and analytics for WordPress. Generates llms.txt for AI crawler discovery, logs AI bot visits, and provides comprehensive analytics dashboard. Usage After activation, the plugin will automatically start generating the llms.txt file and log visits from AI-based crawlers. The crawler logs are stored in the database for analysis. You can view analytics by navigating to LLMS Crawlers in the WordPress admin menu. Admin Analytics The admin panel will display a list of the most recent crawler visits, showing the following information: – Path – URL – Crawler Name – Timestamp Support If you encounter any issues, please feel free to reach out via the plugin support page or create an issue on the GitHub repository. License This plugin is licensed under the GPLv2 or later. New Features Admin Settings Filter: You can now filter active pages, products, categories, and posts for generating the llms.txt. Sitemap Handling: If a sitemap.xml exists, it will be used to get the list of pages. If not, the database will be used as a fallback. SEO Plugin Integration: The plugin integrates with Yoast SEO, All in One SEO, and Rank Math to fetch metadata for the llms.txt file. Frequency Setting: You can choose the frequency for regenerating the llms.txt file (daily, weekly, monthly, hourly). Traffic Capture and AI Agent Logging: A front-end script captures traffic, sends it to the backend, and logs AI agents based on a config.json file. Sitemap Update: The sitemap.xml is automatically updated with the absolute path to the llms.txt and the updated timestamp.

Prime SEO — SEO for the AI Era Prime SEO is an AI SEO plugin for WordPress. Decide which AI crawlers may read your site (GPTBot, ClaudeBot, PerplexityBot and 12 more), publish an llms.txt and llms-full.txt file for the ones you welcome, and see which AI bots actually visit. Classic SEO lives in the same plugin: meta tags, XML sitemap, schema markup, redirects and a 404 monitor. The AI crawler switches work in the free version. Pick a preset (allow all, block training bots only, block all) or set each of the 15 bots by hand in the AI Bots Manager. The robots.txt rules are written for you, and AI Crawler Stats show which AI bots came and which page they opened. Pro: the meta written for you. “Generate All with AI” fills the SEO title, meta description, Open Graph and Twitter fields and the focus keywords of a post in one click; the AI Bulk Generator does the same for every post that still has none. 500 generations a month are included with the license, no OpenAI account needed, or use your own key without a limit. 14-day trial, no card. You always know the next step. A setup checklist on the Dashboard shows what is done and what is left, ticks steps off by itself as you configure the plugin, and links to the right screen. One SEO score, the same number in the editor, the posts list and the Dashboard, points to the posts that need work. Moving from another SEO plugin? Titles, descriptions and redirects are imported from Yoast SEO, Rank Math, All in One SEO and SEOPress, with a backup made first. Documentation | Support | Pro AI Search Optimization — Be Found by ChatGPT & Perplexity ChatGPT, Perplexity, Claude and Gemini read websites with their own crawlers. Prime SEO shows you which of them visit and lets you decide what each one may read. LLMs.txt Generator — Create an llms.txt file automatically and tell AI crawlers what your site is about LLMs-full.txt — Provide full content for deeper AI indexing AI Bots Manager — Allow or block 15 AI crawlers with robots.txt rules added for you: GPTBot, ChatGPT-User and OAI-SearchBot (OpenAI), ClaudeBot, Claude-User, Claude-SearchBot, Claude-Web and anthropic-ai (Anthropic), PerplexityBot and Perplexity-User (Perplexity), Google-Extended (Gemini training), Applebot-Extended (Apple), CCBot (Common Crawl), Bytespider (ByteDance) and Meta-ExternalAgent (Meta) Quick Presets — Allow All, Block Training, Block All, or Custom rules AI Crawler Stats — See which AI bots visit your site, how often, and which pages they read. Visits are logged for 15 crawler user agents, including cohere-ai and Amazonbot AI Visibility Score — See your AI search readiness at a glance AI Content Score — Detailed analysis of how AI engines view your content (Pro) What is llms.txt? → SEO Metabox with Real-Time Analysis Optimize every post with instant feedback and AI assistance: Visual SEO Score (0-100) with circular progress indicator Live Google Preview for desktop and mobile Character counters with color-coded progress bars Multiple focus keywords per post Content recommendations as you type Generate All with AI — One click fills SEO title, description, social meta, and keywords (Pro) The metabox appears in every post and page editor, giving you everything you need to optimize content without leaving the screen. Prime SEO Pro — AI-Powered SEO Prime SEO Pro adds AI automation to your SEO workflow. ⭐ Generate All with AI — Complete SEO in One Click (Pro) Why fill fields manually when AI can do it all? One button generates everything: SEO Title — Optimized for Google (60 chars) Meta Description — Click-worthy snippet (155 chars) Facebook Title & Description — Social-optimized OG tags Twitter Title & Description — Twitter-specific formatting Focus Keywords — Auto-extracted from your content (top 5) 6 fields + keywords in one click. Search and social meta are generated together, so you do not have to fill each field separately. Works with any post type 500 generations a month are included with a Pro license (GPT-4o mini through the Prime SEO AI service, nothing to set up) Prefer no limit? Switch to your own OpenAI key and pick the model; you pay OpenAI directly AI Bulk Generator — Optimize Your Entire Site (Pro) Have 50, 100, or 500 posts without SEO? No problem. AI Bulk Generator finds all posts missing SEO data and generates titles and descriptions for all of them: Smart Detection — Finds posts missing title, description, or both Batch Processing — Generate SEO for hundreds of posts Progress Tracking — Real-time progress with pause/resume Undo Support — Restore previous values if needed Additional Pro Features: AI Content Score — Detailed analysis for AI search optimization Video Sitemap — Auto-detect YouTube, Vimeo, Wistia, Bunny Stream Manual Video Fields — Complete control over video metadata Priority Support — Direct email support Get Prime SEO Pro → Redirect Manager Professional redirect handling: 301 (Permanent), 302 (Temporary), 307 redirects 410 (Gone) and 451 (Unavailable for Legal Reasons) status codes Regex pattern matching with capture groups ($1, $2) Query string preservation for UTM parameters Hit counter for tracking redirect usage Fallback redirect for all 404 errors Import and export functionality 404 Error Monitor Track and fix broken links: Automatic logging of all 404 errors Referer and User Agent tracking One-click conversion to redirect Bulk delete options XML Sitemap Automatic, standards-compliant sitemaps: Sitemap Index with separate post, page, category, and author sitemaps Image Sitemap for Google Image Search Video Sitemap — Auto-detect YouTube, Vimeo, Wistia, Bunny Stream (Pro) Custom Post Types and Taxonomies support Respects noindex settings automatically Beautiful XSL styling for human-readable sitemaps Schema Markup (JSON-LD) Structured data for rich search results: Site-wide: Knowledge Graph — Organization or Person WebSite schema with SearchAction Social profiles (Facebook, Twitter, LinkedIn, YouTube, etc.) Content: Article, BlogPosting, NewsArticle FAQ and HowTo (auto-detected from content) Recipe and Event schemas Product schema for WooCommerce LocalBusiness with 40+ subtypes (Restaurant, Hotel, Dentist, and more) Additional Features Robots.txt Editor — Visual editor with 5 ready-made templates Image SEO — Auto-generate ALT and TITLE attributes External Links Manager — Auto-add nofollow and open in new tab Breadcrumbs — Shortcode and PHP function with Schema.org markup Instant Indexing — Submit URLs via IndexNow API to Bing, Yandex and other IndexNow search engines, by hand or automatically on publish Webmaster Verification — Google Search Console, Bing Webmaster Tools, Yandex, Pinterest, Baidu and Ahrefs verification codes in one place Import/Export — Full backup and restore with automatic backups One-Click Migration Import your existing SEO data: Yoast SEO (Free and Pro) Rank Math (including redirects) All in One SEO SEOPress All titles, descriptions, focus keywords, canonical URLs, and robot settings transfer automatically. Setup Wizard Get started in minutes with the 4-step configuration wizard: Homepage SEO settings Feature toggles (Sitemap, Image SEO, External Links, 404 Monitor) Robots.txt configuration Ready to optimize Built for Performance Lightweight code following WordPress Coding Standards Assets load only on relevant pages No external API calls except optional IndexNow and the optional deactivation survey All data stored in your WordPress database Compatible with caching plugins Developer Friendly Clean, well-documented codebase Hooks and filters for customization WordPress REST API compatible Translation ready Privacy Policy Prime SEO stores all data locally in your WordPress database: SEO settings and meta data Redirect rules 404 error logs (IP addresses stored for debugging purposes only) Backup files External Services This plugin connects to external services only when you explicitly use certain features: IndexNow API (Free) Service: IndexNow (api.indexnow.org) When: When you click “Submit to IndexNow” or enable auto-submit Data sent: Your site’s URLs Purpose: Request faster indexing by search engines Privacy: IndexNow Privacy Policy Prime SEO AI service (Pro only, default) Service: prime-seo-plugin.com/api/ai/ (our own website), which relays the request to OpenAI with our key When: When you use “Generate with AI”, “Generate All with AI”, the AI Bulk Generator or the AI Content Score with the “Included with Pro” option selected in AI Settings Data sent: the post title, excerpt and content (up to 500 words) and the focus keyword; your Freemius install id and license key so the service can check the license; the site address for that check only Stored on our side: per-license counters (how many generations this month) and a log line per request with the license id, install id, status, token counts and duration. Prompts, generated texts, site addresses and IP addresses are not stored Purpose: generate SEO titles, descriptions and social meta within the 500 generations a month included with Pro Terms: Terms of Service Privacy: Privacy Policy OpenAI API (Pro only, with your own key) Service: OpenAI (api.openai.com) When: Same features as above, when “My own OpenAI key” is selected in AI Settings Data sent: Post title, excerpt, and content for SEO generation Purpose: Generate SEO titles and meta descriptions Privacy: OpenAI Privacy Policy Note: uses the API key you saved; no data goes through our servers Prime SEO feedback (optional, free version) Service: prime-seo-plugin.com (our own website) When: Only if you pick a reason and press “Submit & Deactivate” in the dialog shown when you deactivate the plugin. “Skip & Deactivate” and “Cancel” send nothing Data sent: the reason you selected, your optional comment, the Prime SEO, WordPress and PHP versions, and the site language. No site address, no email address, no personal data. IP addresses are not stored Purpose: to learn why people stop using the plugin and fix those problems Terms: Terms of Service Privacy: Privacy Policy No tracking or analytics are performed. No data is shared with third parties except as described above. Support For support, please use the WordPress.org support forum. We aim to respond within 24-48 hours. Credits Prime SEO is developed by the Prime SEO team. Built following WordPress Coding Standards. Icons: Dashicons