It is important to work with accurate data when doing keyword research. There are two reasons for this. First, if you are doing SEO for client websites, particularly if you’ve been assigned to do keyword research using Google’s Keyword tool, it is important that your search traffic predictions are similar to the actual results. Customer expectations are mostly based on traffic and conversions, so it’s valuable to know the accuracy of the tool you are using.
Second, estimated sales based on conversions also use search query volume data, which can be taken from Google’s Keyword tool. If the data you are gathering is highly inaccurate, then you will come up with some unrealistic estimates for sales and conversions.
Scope and limitation of this study
You need to determine first what sets of data need to be compared. Suppose you focus on global or worldwide results as opposed to local search results. In Google’s Keyword tool, you can clearly see the “Global Monthly Searches” column. See screen shot:
According to Google, this is the estimated 12-month “average” of user queries in Google’s search engine. Meanwhile, the “impressions” data in Google Webmaster Tools shows the exact volume of search queries done for that specific keyword, where your website is ranking in Google search results.
But note that the Google Webmaster Tools impression data is NOT a “12-month average;” it only applies to the latest/current month (e.g December 10, 2010 to January 9, 2011). You might be aware of the “trending” and “seasonal” effect on search volume queries. So you can expect some errors based on this effect.
This is one limitation of this study, since you cannot do a “one-to-one” comparison between Google’s Keyword tool data and Google Webmaster Tools data using the same time frame. For example, if you intend to gather the November 2010 data from the keyword tool, you cannot get the equivalent November 2010 impression data with Google Webmaster Tools because it will display only the current/active one month set of data.
Likewise, if you use the latest impression data, you cannot get the equivalent "latest month" data with Google’s Keyword tool, because the latest they can provide is the November 2010 data (if you are gathering data in January 2011). See screen shot below:
In this study, you can only compare the global monthly search volume (12-month average) to the latest one month impressions data available in Google Webmaster Tools. Another factor considered in this study is the “match types.” Google’s Keyword tool offers three match types to get monthly searches data. These are the "broad," "exact" and "phrase" matches.
Some SEO practitioners often used the “exact” match type, as this appears to be closer to the exact data. But no conclusive test has been made advising that one should use either an “exact,” “phrase” or “broad” match to get more accurate estimates of actual search volume queries. This factor is considered in this study.
To test, the following steps are employed:
1. Select a test website with a Google Webmaster Tools account and lots of ranking keywords (main + long tail keyword). Select only those keywords that are ranking on the first page of Google (average ranking is less than or equal to 10).
2. Gather the global monthly search volume for those keywords using Google’s Keyword tool. Gather one set of data for each match type: exact, broad and phrase.
Since global monthly data is needed, set countries to “All countries” and languages to “English.” Download or export the data to an Excel spreadsheet.
3. Finalize all of the data needed in the analysis in an Excel spreadsheet. Include the Google Webmaster Tools impression data. Tabulate the data according to this screen shot:
4. Finally, once all the data is completely filled in, you can compute the percent error. Percent error is a measurement of accuracy. The lower the percent error, the better.
For the best study on the accuracy of the tool, the percent error for each match type is computed. The formula for percent error is as follows:
Percentage error = [(estimated data – actual data)/actual data] x 100%
The percent error should always be positive, so the absolute value is taken from the result. In this study, the “actual” data comes from Google Webmaster Tools, since it is taken from actual results. The “estimated” data comes from Google’s Keyword tool.
For example, the percent error formula for the “exact” match type is as follows;
Percentage error = [(Google keyword tool exact match – GWT Impression data)/GWT Impression data] x 100%
Analysis and Conclusion Concerning Results
You can download the data analysis used in this study here: http://www.php-developer.org/wp-content/uploads/tutorials/GoogleKeywordAccuracy.xls.zip. The number of samples used in this study is 96 (number of keywords).
The first thing to note is that the percentage of error is very high. According to the standard rules for applying percentage of error in accuracy measurements: http://www.isixsigma.com/index.php?option=com_glossary&id=234, the maximum acceptable percent error is only 30%.
Second, out of the three match types, only the exact match has the lowest percent error (210%).
What are the possible reasons for the high percentage of error? First, keep in mind that keyword search volumes are seasonal, and the actual data (at Google Webmaster Tools) cannot be compared exactly in the same time frame as Google’s Keyword tool. This limitation was already discussed in detail in the previous section.
Second, the actual impression data can change real time in Google because of the changes in keyword ranking. Although the keywords selected in this study are primarily keywords ranking on Google’s first page, the ranking may change in real time. This means that the keyword may go lower than the first page at some point, and receive fewer, or even no, impressions (at least temporarily).
It is because of these ranking fluctuations that Google Webmaster Tools uses averages and shows the “average” ranking.
Based on the results, it is not recommended that you rely solely on Google’s Keyword tool for predictions that require high accuracy and sensitive computations, such as sales. Customer expectations set using this raw data can be difficult to achieve, because of the high uncertainty of the predicted results when compared with the actual impression figures.
You can, however, use Google’s Keyword tool as a “rough” guide or estimate, but it cannot be relied on by itself for making final financial decisions. In this case, it is recommended that you use the actual figures of search queries, such as those found in Google Webmaster Tools.
To provide much more accurate estimates, it is recommended that you use the “exact” match type in getting search volume figures that coincide closely with the actual impression data.
Finally, in keyword research, you can rely on Google’s Keyword tool to determine those keywords that are good to target by simply looking at the search volume data. It cannot, however, be relied upon to provide accurate projections in terms of sales and actual traffic.