io.github.sharozdawa/robotstxt-ai
by io.github.sharozdawa · Added 9d ago
Install
git clone https://github.com/sharozdawa/robotstxt-aiAbout
Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.
Tags
ai-mlmcp-serverofficial-registry