Skip to content
Utilora

Robots.txt Generator

Build a valid robots.txt with allow and disallow rules, crawl-delay and sitemap links.

Webmaster ToolsRuns in your browser

Results appear here as you type.

How to use

  1. Choose which crawlers the rules apply to.
  2. Enter one path per line, starting with /.
  3. Save the output as robots.txt in your site root.

More tools