Polite Bots - Meta Tags and Content Values
(Page 3 of 4 )
According to Vanessa Fox, if you stuff your meta tags with conflicting values, such as putting index and no index, Google always follow the most restrictive value.
<META NAME="ROBOTS" CONTENT="NOINDEX">
<META NAME="ROBOTS" CONTENT="INDEX">
In the above, the "noindex" value will be followed (the most restrictive). Also if the above meta tag is written as
<META NAME="ROBOTS" CONTENT="NOINDEX" CONTENT="INDEX">
Google still follows the most restrictive value. Vanessa Fox also mentioned that Google "recommends" that you place all content values in one meta tag. This makes it easier for the Googlebot to read the values and reduces "chances of conflict." However, whether you have one meta tag containing your content values or you have two, Google aggregates and reads them alike.
Conflict Resolution
If the meta tag and the robots.txt clash, as they would in situations where you don't exclude a file in the robots.txt file but then exclude it in a meta tag in the file, Google still follows the most restrictive value. In this case, that would be the meta tag, so note that once a file is blocked in robots.txt it is never crawled by the Googlebot. Some valid meta tag values are:
- NOINDEX - Prevents a file in a website from being indexed.
- NOFOLLOW - Prevents the Googlebot from following any links on the page. (Note that this is different from the link-level NOFOLLOW attribute, which prevents Googlebot from following an individual link).
- NOARCHIVE - Prevents the web page from being cached.
- NOSNIPPET - Prevents any description from appearing below the page listing in the SERPs; also prevents the page from being cached.
- NOODP - Prevents the Open Directory Project description of the page from being used in the description that appears below the page listing in the search results.
- NONE - Equivalent to "NOINDEX, NOFOLLOW."
Note that the "NONE" content value means "NOINDEX", "NOFOLLOW" and that if it is included in your meta tags, your page won't get crawled at all.
<META NAME="ROBOTS" CONTENT="NONE">
You pretty much exclude ALL the bots when you put the "none" value in your Meta tags. So much for exclusion protocols; let's see if there are ways to get the Googlebot to come over.
Next: Is There Any Need to Trigger Google's Bot? >>
More Website Submission Articles
More By Akinola Akintomide
