seo
Processed locally in your browser

Robots.txt Generator

Generate standard robots.txt files for search engine crawlers with allow/disallow rules.

/api//admin/

How to Use Robots.txt Generator

1

Configure Directives

Select default permissions and add paths to block (e.g. /admin/, /private/).

2

Add Sitemap

Enter your full sitemap URL (e.g. https://example.com/sitemap.xml).

3

Download

Download the file and place it in your site's root directory.

Features & Capabilities

Pre-configured profiles for Googlebot, Bingbot, and universal (*)
Add custom Disallow and Allow directory rules
Include XML sitemap URL directive
One-click copy or direct file download as `robots.txt`

Frequently Asked Questions

It must be placed at the absolute root of your domain: `https://example.com/robots.txt`.

Related Utilities