I had never thought about this before, but I definitely started thinking about it since then. Logically, it would make perfect sense that the search engines might completely disregard the meta tag information, and determine a page’s relevancy score based solely upon that page’s content. After all, with all the abuse that the meta tags have sustained over the years, the best way to eliminate the spam issue is to ignore them completely. But I think the concept is far more complicated than that. What about those sites that intentionally misuse their meta tags to spam the search engines? Do Google and Yahoo take that into account when determining relevancy score? If so, then that means the search engines are not entirely disregarding the meta tags, because they certainly apply penalties for misuse of meta tags. Or do they? Further, if a web page is missing its meta tags, how will the search engines that still read them, like MSN, treat those pages?
I could sit around all day and speculate, but the proof is in the pudding, so to speak. So our firm decided to perform an experiment. We chose six websites to use as our test group, and six others as our control group. In the control group, we used naturally optimized sites that included meta tags. In the test group, we utilized the same optimization techniques on the sites, with the exception of the meta tags, which we left out of the page completely.
We knew we were going to have to wait for a Google update, so we wanted to watch these sites for at least six to eight weeks. We analyzed statistics, keywords, page content and everything else that a good SEO firm should monitor when they are assessing the performance of websites. We treated the test group no differently than we treated the control group. Our only variable among these two groups of sites was the absence of meta keyword tags and meta description tags in the test group.
When I was brainstorming as to why the lack of meta tags would affect search engine results, and how drastic the change would be, I thought of a couple of theories as to why this might happen.
I mentioned the first one earlier: abuse of the meta tags in the past has led to complete disregard of them by a search engine spider. We have seen less and less emphasis placed on the meta keyword tags by the search engines, and even though the description tags seem to be listed in the SERPS under a particular URL, we also realize that the description can easily be different than the actual site content itself. Over the past few months, we’ve seen the relevancy of the description meta tag deteriorate very rapidly.
Consider this: search engines are listing more than just HTML files in their SERPs. They are listing Adobe PDF files, XML files, including RSS feeds, and with the help of a Macromedia Software Development Kit (SDK) for search engines (or similar software), a search engine can now read Flash movies and websites. With all of this new technology, it only seems reasonable that a search engine could find relevant pages with these other web languages. One thing you will notice, however, is that none of the web languages I’ve listed utilize meta tag information.
Google’s search engine, in particular, has refined the technique of extracting content from web pages, XML files, PDF files, and even Flash movies. It only makes sense that Google will turn this same method on HTML pages to find a pages’ relevancy score.
I touched on a few questions that came to mind during our brainstorm sessions earlier. And certainly we came up with a few very valid and important questions.
- What does a search engine use to replace meta tag information if it is not present?
- How does a search engine that uses meta tags treat web pages that don’t have meta tag information?
- How much of a difference does eliminating the meta tag information truly make?
- Do Google and Yahoo still take meta tag information into account in order to penalize for spamming?
- Which search engines still use meta tag information, and which do not?
The Results of the Experiment
After the latest Google update in March, we compared the results of our experiment, and came up with some very interesting observations. Our research indicated first and foremost that our theory was far more complex than I stated, but essentially it turned out to be fairly accurate.
Based on our results, I will soon answer the questions that I asked earlier in this article. Granted, our small experiment probably in no way determines any definite behaviors of the search engines in regards to meta tag information, or the lack thereof. This experiment is to simply show trends in what directions these particular search engines may be heading. And analysis of trends seems to be at the heart of SEO these days. I will take a look at the changing tides of SEO in a later article.
Essentially we optimized all of these sites being examined in the course of the experiment in exactly the same ways, with careful research on content keywords, competition, backlinks; the whole SEO works. The major difference between the sites in the control group and the sites in the test group were the presence and absence of the meta keyword and description tags respectively.
Since the experiment, we have also been analyzing our competitors’ sites to see what they are doing, and whether meta tags or the lack thereof are affecting their PageRank as well as their SERPs positions, and comparing the trends in a similar fashion to our experimental sites. It seems honestly to make no difference whatsoever that a site may use meta tags, or not use them at all. We DO see, however, that a huge number of meta keywords or stuffed keyword tags, or any other spam areas ARE being assessed and penalized.
The Answers to Our Questions
- “What does a search engine use to replace meta tag information if it is not present?” We have discovered that the search engine utilizes the title tag and the actual content of the page being spidered. Clever, huh?
- “How does a search engine that uses meta tags treat web pages that don’t have meta tag information?” The same way that a search engine that doesn’t use meta tags: they pull the information from the title tags and the page content.
- “How much of a difference does eliminating the meta tag information truly make?” Overall, there does seem to be a slight difference in the way that these pages without meta tags are indexed. Of the sites in our experiment, we found that many of them actually ranked higher in position of the SERPs than the sites that utilized meta tags. And while our research is still in its infancy, we do have to consider that our sites may have had additional issues that may have made the results look like the sites without meta tags did better, which in actuality it could be purely coincidence. And as hard as I find this to believe, I have to admit that this could indeed be a possibility.
- “Do Google and Yahoo still take meta tag information into account in order to penalize for spamming?” Yes, they do. So while general meta tags seem to be disregarded, they are still watched for keyword stuffing and other spamming techniques.
- “Which search engines still use meta tag information, and which do not?” This is not as black and white as we really hoped it would turn out. MSN has been shown to utilize both the keyword and description tags as part of their relevancy score to determine SERPs position. Yahoo doesn’t use the keyword tags in their relevancy score as far as we can tell lately, but will use the description tag. Google uses neither keyword or description tags in their relevancy score, and we believe, although this is a definite gray area, that Google disregards both of them in determining overall PageRank.
In conclusion, we have determined that meta keyword and description tags do not make one bit of difference in the ranking positions, as well as no difference in the SERPS pages or PageRank, unless they are considered spam. They certainly will not either make or break a site unless that site is spamming the search engine.
It is my firm belief, even before this experiment, and of course has now has been solidified, that meta tags were nearly deprecated in the search engines, with the exception of perhaps MSN. Still, the differences that we’ve seen at all in MSN were almost inconclusive in our experiment’s results.
We will honestly have to watch the sites in the control and test groups over the course of several updates to have the most accurate trends, of course, however for the sake of argument; we wanted to at least alert you to this grand phenomenon.
I’m afraid in the face of the changing SEO industry that those SEOs that will rely heavily on their meta keyword and descriptions tags are going to find that more and more they will be fairly disappointed in the effectiveness of optimized keyword and description tags. This will particularly upset those that have really no other ideas as far as page optimization, and several of those in the industry will have to rethink their entire strategy and optimization tactics. But relax; you still have MSN to love your optimized meta keywords and description tags.
For information about Jennifer Sullivan Cassidy’s professional search engine optimization services, please visit her site at First Class SEO.