Block AI Crawlers
Protect Your Content from AI Scraping This plugin tells AI companies not to crawl your site for their products via standard robots.txt rules. Features Blocks AI Bots and Crawlers The plugin blocks dozens of AI bots via your site’s robots.txt file. You can toggle the blocks for each individual crawler. Blocked crawlers include: OpenAI – Blocks crawlers used for ChatGPT Google – Blocks crawlers used by Google’s Gemini AI products Meta – Blocks FacebookBot and Meta training crawlers Anthropic – Blocks crawlers used by Claude Perplexity – Blocks crawlers used by Perplexity Apple – Blocks Applebot-Extended Amazon – Blocks Amazonbot …and 150+ more via ai.robots.txt The blocked crawler list is generated at build time from ai.robots.txt (robots.json, MIT). See the plugin’s README.md for license attribution. Experimental Meta Tags The plugin adds the experimental “noai, noimageai” directive to your site’s meta tags. You can toggle this on and off as needed. Custom robots.txt Rules Have custom entries for your robots.txt file? You can now add them directly through the plugin! Usage After activation, the plugin will automatically update your robots.txt and add the necessary meta tags. No further configuration is required, but you can check the settings page for a full list of blocked crawlers. Limitations While this plugin aims to block specified crawlers, it cannot guarantee complete protection against all forms of scraping, as some bots may disregard robots.txt directives. Support For questions or support, please post on the forums.
CodingBunny LLMs.txt Generator
CodingBunny LLMs.txt Generator creates an advanced llms.txt file to help AI systems discover and understand your content with rich semantic metadata. Key Features AI Metadata: Schema.org types, canonical URLs, related content, differentiated licenses Smart Filtering: Decorative image detection, featured-only option, configurable limits Crawler Control: Allow/Disallow directives, AI bot allowlist, SEO noindex integration Rich YAML: Title, category, type, date, summary, author, language, tags, priority Automation: Daily/weekly scheduling, immediate on-publish, generation history Developer-Friendly: 20+ hooks/filters, WooCommerce support, multilingual ready Schema.org auto-detection (Article, Product, ContactPage, FAQPage, etc.) Canonical URLs (Yoast/RankMath compatible) Related URLs (parent/child pages, related posts/products) Differentiated licenses (content vs. media) HTML entity fixes and improved text normalization
