Robots.txt Generator - SEO Tools - Search Engine Optimization, Google Optimization

SEO Tools - Robots.txt Generator

Google is making some changes on how automated search results are handled, and it is causing some of our tools to not operate correctly. We apologize for any inconviences this has caused, and we are working on ways to make our tools functional again.

Use this tool to generate a simple robots.txt file for your website. This file allows you to hide files or directories that you don’t wish the search engine spiders to find. This generator tool is designed to create the text for the file for you; you can then make changes afterward should you deem it necessary.

Generating a robot file with this tool is ideal if you wish to block certain directories or files from search engines.

To use the generator tool, enter the required information below and click the button. You will then be shown the text for the file. Copy this to a file called robots.txt and place on the root of your website (in the same place as your home page).

Imposing Restrictions
You may impose restrictions on which web pages to disallow indexing. By default, most users will want to allow all directories except their /cgi-bin/ directory, which commonly holds scripts, and their images directory /images/. To enable all web pages, select Yes to "Enable All Webpages," then enter each web page or directory path in the exclusion box, one per line.

Example: "http://www.yourdomain.com/cgi-bin/" (Excludes the /cgi-bin/ directory)
Example: "http://www.yourdomain.com/images/" (Excludes the /images/ directory)
Example: "http://www.yourdomain.com/welcome.html" (Excludes the /welcome.html web page)

Add the Robots.txt Generator to your web site FREE!


	Robots.txt Generator Tool © SEO Chat™ Allowed User Agent Select user agent or use default for all agents Enable All Webpages Yes No Exclude These URL's Enter URLs you wish to exclude /cgi-bin/ /images/ Enter Captcha To Continue To prevent spamming, please enter in the numbers and letters in the box below Report Problem with Tool.