Free · robots.txt

Robots.txt Generator

Build a correct robots.txt with crawl rules and your sitemap. Wrong directives can deindex a site — this keeps it valid.

Preset

User-agent

Disallow (one path per line)

Allow (one path per line)

Crawl-delay (optional)

Sitemap URL

robots.txt

User-agent: *
Disallow:

Upload as /robots.txt at your domain root.

JW Digital audits crawl rules, indexation and technical SEO so your pages are found — not accidentally blocked.

More free SEO tools

Generate the sitemap.xml you reference in robots.txt.

Push URLs to search engines for faster indexing.

Add JSON-LD structured data for rich results.