Cómo usar
Select a User-agent, add Allow/Disallow rules, and generate robots.txt. You can also set Sitemap URL and Crawl-delay.
- User-agent selection (*, Googlebot, Bingbot, etc.)
- Add Allow/Disallow rules
- Sitemap URL setting
- Crawl-delay setting
- Preview + Copy
Casos de uso
- When creating robots.txt for a new site
- When setting crawl rules visually
- When handling multiple User-agents