The Five Most Important Things You Can Do with Google Webmaster Tools - Fourth and Fifth Important GWT Tasks
(Page 4 of 4 )
FOURTH MOST IMPORTANT THING: Check your robots.txt and submit an XML sitemap
Robots.txt helps to prevent duplicate content and maximizes bandwidth in your site. By navigating to Dashboard -> Sitemaps (for sitemap XML) and Dashboard -> Analyze robots.txt, you can upload an XML sitemap and test your robots.txt before uploading to the root directory of your server.
For more details on robots.txt analysis, you can read this article.
FIFTH MOST IMPORTANT THING: Remove URLs from your site
In any website, there are pages that are unimportant and not useful for both visitors and search engines. In this case they can simply be blocked in robots.txt, and then you can remove that URL in Google Webmaster tools. The overall purpose is to have a clean index. A clean index in Google means that crawling will be efficient and Google will not revisit those unimportant URLs.
To remove URLs, you can go to Dashboard -> Tools -> Remove URL
Make sure you have followed the requirements before you remove the URL, which can be any of the following:
- The URL will be blocked in robots.txt.
- The URL will be placed with meta no-index tag.
- The URL returns a 404 header status.
Other than that, Google will deny your request. So make sure to double check things first before removing any URL.
| DISCLAIMER: The content provided in this article is not warranted or guaranteed by Developer Shed, Inc. The content provided is intended for entertainment and/or educational purposes in order to introduce to the reader key ideas, concepts, and/or product reviews. As such it is incumbent upon the reader to employ real-world tactics for security and implementation of best practices. We are not liable for any negative consequences that may result from implementing any information covered in our articles or tutorials. If this is a hardware review, it is not recommended to open and/or modify your hardware. |
/ 16 
