Google Moves to Continual Indexing

If you’ve noticed that Google seems to be updating their index more frequently than once a month, you’re not alone. Clint Dixon observed this as well. In this article he discusses how this changes the world for website owners, search engine optimizers, and anyone else trying to get on the front pages of popular search engines. took a step forward in offering the most relevant information available by beefing up their mission statement. For a long time Google has updated their database of information by indexing the World Wide Web once per month. In light of recent events I have come to the conclusion that Google is now updating their index much more frequently, as evidenced by server statistics from my clients, and from my own server.

With the November 2003 Florida Update, Google dropped many of the offending webmasters who employed black hat SEO techniques off of the SERPs (Search Engine Results Pages). This upset a great many webmasters whose sites sat on the front pages of for years, and who felt smug without preparing for the future.

Google needed the Florida update in order to convince investors that “spammy Web sites” would no longer rank on the front pages of their searches, in preparation for the company’s IPO. Since that time Google has gone public, and its shares of stock have risen in price significantly. This brought up Google’s next move, one that also hurt webmasters. These webmasters weren’t using black hat SEO tactics, but instead were buying, trading, and selling links with other websites–in fact, entire websites were built for brokering links. The latest Internet Back Link Update allowed Google to check the links from sites and rid its ranks of websites trying to manipulate their Google Page Rank Technology using questionable tactics.

Other updates included Google’s technological steps, such as having bots sit, so to speak, on servers continually looking for new fresh content. Recently, while speaking with a client about getting their website listed on the front pages of Google, I did a search for one of their keywords. (Ironically enough, it was the word “website.”) What I found startled me was very different from what I remembered, since for years this word returned mostly those who designed or built websites. The notable changes were the two new websites for George Bush and John Kerry. I can only imagine the faces of the webmasters whose websites were knocked off the SERPs by the candidates for president.

{mospagebreak title=A Revolution in the Making}

This move toward continuous indexing will revolutionize search, and the businesses of those of us who strive to get clients listings on Google’s front pages.

How will Google’s decision to index websites more frequently change the way search engine optimization is performed? SEO will become less of a need in order to obtain front page listing results, and search engine marketing will grow to be the main focus for marketing and optimization firms, and for website owners.

The days of putting up a site and coming back once a month to update the website and keep the spider bots happy are gone for good. We live in the age of immediacy, and because of this the search engines are working to establish results that are the most current and therefore most relevant to users’ search queries. The days of submitting your website and waiting more than thirty or sixty days just to get a listing are history. For the SEO, this means it’s time to change the focus of his or her business; for the website owner, this can be a blessing–and a curse.

If you have been paying attention to your stats, it would almost appear that the googlebot is sitting on your server. In a way it is! In order for Google to provide relevant results, those results must be current. Returning results that are already a month old is not a smart, long term business model if Google plans to stay in business. Advertisers are looking for performance, and they don’t care what worked in the past; they want immediate results for their advertising spend.

{mospagebreak title=Evidence of the Change}

I came upon this while speaking to a friend in British Columbia in early October 2004 about the possibility of getting his website listed on Google’s front page natural results listings. His chosen keyword term was, again, “website.” This made me laugh a bit, knowing all the competition and how entrenched this keyword term was for the many designers and developers who had resided on the front page of Google for so long. Any idea of moving most of them was funny. Or so I thought. When I typed in the term and saw the results I sat and felt my jaw hit the floor. The listings had changed!

What was now on the front page was a shock. Two new entries had found their way to the front-page results, the web pages of both George W. Bush and John Kerry, the two candidates running for the office of President of the United States at the time. Why were they listed so highly for the word “website” when these sites had nothing to do with the search term, other than the fact that each was a website?

This made sense to me when I thought about what people were searching for at the time. The terms I figured were being used were “bush website” and “kerry website,” meaning that the two politicians, by virtue of searches, would show up for “website.” Millions of people were typing that into their queries about each candidate. To Google, these two websites were the most current and heavily searched terms; therefore the websites belonged in the front page results for search queries using “website.”

The blessings come in many ways, with the curses probably outweighing the blessings. Those who are in the search marketing business, and who do not know how to keep sites current and relevant will soon be among the unemployed. Those website owners who use software to optimize their sites and believe that is enough to keep them on any search engine’s front page for long will soon learn different. No longer will SEOs be able to sit down, do some keyword research, optimize the coding and content, and sit back and watch each month as the client’s site sits on the search engines’ front page results.

{mospagebreak title=What To Do About the Change}

SEO consultants and marketing firms will need to move their clients to updating their websites more frequently and with relevant content, or they will need to implement this strategy into their services offerings. Additional strategies will need to be implemented in moving clients onto the front page of the major search engines.

Another area where freshness and relevancy are coming into play is in the reciprocal links that so many mistakenly think are the basis for good search engine results listings. Judging from what Google did during the Internet Back Link Update last month, any links that are developed will need to be fresh and relevant as well. The thinking behind this is based on Google’s current strategy of presenting fresh, relevant content. Links that are several months old will no longer have as much relevancy or weight as links developed recently. Google treats links like votes. Each link is a vote for the page to which it links. Using rational thinking, votes from the past do not count in the present. The votes that George Bush received in 2000 do not count, and are irrelevant to, the Presidential election in 2004. Therefore links that are old and outdated will most likely have little value after a month or two.

The future of search engines and how your website will rank in the search engine results pages will be determined by popularity as well as the steps involved in search engine optimization. This goes hand in hand with our current lifestyle and its extreme breakneck speed. What is popular today will undoubtedly be out of fashion soon. If your website content or links are stale, you will not rank as well in the search engines as a site that stays current by presenting new or changed content frequently.

Due to the changes at hand in the search engines, (MSN states it will crawl daily and weekly) the work that search engine optimization specialists perform now will need to change dramatically. No longer will keyword density and weighting inside the content matter as much as how new the content is presented, and how relevant it is to your overall website theme.

I recently read in a forum a particularly harsh dissertation on how Google destroyed businesses by dropping websites during its notable indexing updates, such as the Florida update, where a great many websites that had previously employed black hat SEO tactics as well as overt spamming and other manipulations of the search results to gain the front pages results listings on Google, were dropped from Google’s database or sent so far down the listings that they would not be back anytime soon. I had to think for a minute before responding. Would people build a business based on the assumption that their website would live eternally on Google’s front page? It then occurred to me that my business is built on getting websites to the front page of Google. And in that thought I went on to answer the post, “If anyone builds a business solely dependent on front page results listings from Google, they don’t deserve to be in business.” It is never a good idea to place all your eggs in one basket.

No matter which business you are in, whether you’re the webmaster who runs another’s website, a website owner who handles day to day operations, a search engine optimization specialist, or an advertising agency, your business will need to redevelop itself to one that stays fresh in technique, presentation and the overall strategies employed.

[gp-comments width="770" linklove="off" ]