AI Crawler and SEO Manager by akbansa
The AI Crawler and SEO Manager by akbansa plugin generates an llms.txt file for crawler discovery and logs visits by AI-based crawlers such as GPTBot, ClaudeBot, and others. It also provides an analytics page in the WordPress admin dashboard to view crawler activity by path, URL, and timestamp. Key Features: Generates llms.txt file for crawler discovery. Removes llms.txt on plugin uninstall Schedules automatic generation of llms.txt daily at midnight (frequency configurable: daily, twice daily, hourly) Uses atomic file writes for llms.txt and sitemap.xml for safety Integrates with Yoast SEO, Rank Math, or All in One SEO for metadata (falls back to DB if not present) Admin panel to select/exclude categories and content types for llms.txt Logs every AI crawler request from the frontend (user agent, crawl info, page URL, IP, referrer) Filters crawlers using config.json and stores all details in a dedicated DB table Analytics dashboard in admin to view recent crawler activity Robust AJAX security and error handling Detects and logs visits from AI crawlers based on their User-Agent. Stores crawler logs in a database for analysis. Supports popular AI-based crawlers (e.g., OpenAI, Anthropic, GPTBot, etc.). Option to customize the list of crawlers via a config.json file. Short Description Advanced AI-powered SEO and analytics for WordPress. Generates llms.txt for AI crawler discovery, logs AI bot visits, and provides comprehensive analytics dashboard. Usage After activation, the plugin will automatically start generating the llms.txt file and log visits from AI-based crawlers. The crawler logs are stored in the database for analysis. You can view analytics by navigating to LLMS Crawlers in the WordPress admin menu. Admin Analytics The admin panel will display a list of the most recent crawler visits, showing the following information: – Path – URL – Crawler Name – Timestamp Support If you encounter any issues, please feel free to reach out via the plugin support page or create an issue on the GitHub repository. License This plugin is licensed under the GPLv2 or later. New Features Admin Settings Filter: You can now filter active pages, products, categories, and posts for generating the llms.txt. Sitemap Handling: If a sitemap.xml exists, it will be used to get the list of pages. If not, the database will be used as a fallback. SEO Plugin Integration: The plugin integrates with Yoast SEO, All in One SEO, and Rank Math to fetch metadata for the llms.txt file. Frequency Setting: You can choose the frequency for regenerating the llms.txt file (daily, weekly, monthly, hourly). Traffic Capture and AI Agent Logging: A front-end script captures traffic, sends it to the backend, and logs AI agents based on a config.json file. Sitemap Update: The sitemap.xml is automatically updated with the absolute path to the llms.txt and the updated timestamp.
Top keywords
- llms13×2.77%
- llms txt12×2.55%
- txt12×2.55%
- crawler11×2.34%
- file8×1.70%
- admin7×1.49%
- ai7×1.49%
- crawlers7×1.49%
- logs7×1.49%
- seo7×1.49%
- analytics6×1.28%
- llms txt file5×1.06%
UpdraftPlus
The UpdraftPlus Backup & Migration Plugin is trusted by the WordPress community to backup, restore and migrate their WordPress websites. UpdraftPlus is actively installed on more than 3 million websites around the world. Backup with UpdraftPlus UpdraftPlus is the world’s highest ranking and most popular scheduled backup and migration plugin. Backup to your preferred storage location and restore in just three clicks. Backup to Dropbox, Google Drive, Amazon S3 (or compatible), Rackspace Cloud, FTP, DreamObjects, Openstack Swift or email. The paid version also allows you to backup to Microsoft OneDrive, Microsoft Azure, Google Cloud, Backblaze B2, SFTP, SCP, pCloud, WebDAV or UpdraftVault, our integrated storage option for UpdraftPlus. Backup manually or schedule to run every 2, 4, 8 or 12 hours, daily, weekly, monthly or fortnightly. Restore with UpdraftPlus Your WordPress website is vulnerable. Your site may get hacked. Something could go wrong with an update or your server could crash. You might need to reverse a change made or a mistake caused by human error. Whatever the reason, you can restore your WordPress website in just a few clicks with UpdraftPlus. Choose which components to restore (e.g. plugins, themes, database etc.), click again to restore, then click again to return to the main screen. Migrate with UpdraftPlus It’s easy to migrate your WordPress website to another web host, server or domain with this free version of UpdraftPlus. To migrate, simply download your database, plugins, themes etc. from the source site, then upload them into your destination site. When migrating, a built-in search-and-replace engine identifies old strings and replaces these to reflect the new location. UpdraftPlus saves you time and reduces the risk of broken links or missing files associated with manual migrations. Why UpdraftPlus? Writing a reliable backup and migration plugin that ‘just works’ consistently across millions of different WordPress deployments is hard. UpdraftPlus is relied upon to do exactly that. We’re trusted and actively deployed on more websites around the world than any other WordPress backup and migration plugin. UpdraftPlus: Backs up, migrates and restores Offers a huge number of remote storage locations Allows you to schedule backups, so you can ‘set and forget’ Is comprehensive and easy to use Is proven to work on over 3 million sites. UpdraftPlus Premium The free version of UpdraftPlus will back up and migrate your website perfectly well. However if you need more features and options you can purchase our Premium version. Backup and Migrate with UpdraftPlus Premium, and: Get automatic backups before updates. Restore the very latest version if a WordPress or plugin update breaks your site. Backup Incrementally. Changes are added to the master, saving server resources vs repeatedly backing up in full. Get more remote storage options including Microsoft OneDrive, SFTP, Microsoft Azure, WebDAV, Google Cloud, SCP, Backblaze and pCloud. Get 1GB of integrated storage with UpdraftVault for 99.999% reliability, redundancy and scalability. Get more control over how many backups are stored in a specified interval. Restore from other backup plugins including BackWPup, BackupWordPress, Simple Backups and more. Run backups at set times e.g. during periods of low traffic. Backup to more than one location for added protection. Get detailed reports. Includes cryptographic checksums so you can verify the integrity of the backup files and more. Get Premium migration. Migrating to the destination site is more direct and can be done from the source site. Get Multisite / multi-network compatibility. Backup non-WP files and databases e.g. tables belonging to your e-commerce store or customisations to WordPress core. Get database encryption. Manage your backups and migrations from the WP-CLI. Get Premium Support. Find out more on our comparison page. UpdraftPlus Premium is available for purchase here. Managing multiple websites? UpdraftCentral is a powerful remote control dashboard for WordPress that allows you to manage your backups, as well as updates, users, pages, posts, plugins and themes from one central location. Choose between: UpdraftCentral (free, self-hosted) UpdraftCentral Premium (paid-for, self-hosted) UpdraftCentral Cloud (paid-for, fully-hosted) Optimize your sites centrally by combining the power of WP-Optimize and UpdraftCentral or manage your backups centrally by combining the power of UpdraftPlus with UpdraftCentral. Need to create a temporary clone of your site? UpdraftClone makes it quick and easy to create a temporary sandbox for changes you want to try out. Simply select the versions of WordPress and PHP that you want, and we’ll do the rest. More about UpdraftClone The full suite of plugins from Team Updraft and friends UpdraftPlus. Backup, migrate and restore. Rated 5* on the plugin directory and trusted by more than 3 million WordPress website owner WP-Optimize. Clean the database, compress images and caching. Optimize your WordPress website. 5* user rating. 1 million + active installs. All-In-One Security (AIOS). Secure your WordPress website. Comprehensive, feature rich and easy to use. 5* user rating and 1 million + active installs. WP Overnight. Quality Extensions for your WooCommerce store. 5* rated invoicing, order and product management, customer management solutions and more. Easy Updates Manager. Take control of updates. Lots of features for free, and a Premium version with even more – over 300,000 users. Internal Link Juicer.Boost your SEO. Automate internal links within your WordPress website. Save time and rank higher in the search engines. For other useful, free plugins see our lead developer’s profile, here. Are you multilingual? Can you translate? Do you want to help people that speak your language to backup, migrate and restore their WordPress websites? The UpdraftPlus backup, migration and restoration plugin is ready and waiting. The translation process is easy, and web-based – please go here for instructions: https://updraftplus.com/translate/. Or, if you’re already an expert WordPress translator, just pick out the .pot file from the wp-content/plugins/updraftplus/languages/ directory – if you scan for translatable strings manually, then you need to get these functions: _x(), __(), _e(), _ex(), log_e(). Many thanks to our existing translators. License Copyright 2011-24 David Anderson This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA Non-English translators are provided by volunteers, and wordpress.org does not give plugin developers control over their translations; thus relying on them is at your own risk; UpdraftPlus cannot guarantee that translations from the original English are accurate. We recognise and thank those mentioned at https://updraftplus.com/acknowledgements/ for code and/or libraries used and/or modified under the terms of their open source licences.