The History of Search and Search Technology - Index
(Page 2 of 4 )
The index is the place where search engines keep basic copies of web pages and sort search results. When you a do a search, search engines do not search the web; they show results from their index. The number of pages in the index does not represent the entire web, but the number of pages that the spider has discovered, scanned and saved.
The search results count (i.e. Results 1 - 10 of about 160,500) on Google, Yahoo, Live and MSN tells the number of documents in the index that have a search term somewhere in the content or inbound links.
The index is the place where search engineers apply algorithms, and it is the place where rankings are partially determined. Search engineers may choose to apply an algorithm to the entire index, or only to a portion of it.
Datacenters and Different Indexes
Search engines have multiple datacenters around the world. When you enter a search term, your query is directed to the closest datacenter.
Different datacenters may have slightly different indexes, especially during an update. As a result, search results may differ depending on your location. For example, during an index update, Bob in LA will see different results from Tim in New York. This difference in indexes is called the Google Dance, and it's used by SEOs to spot an update.
A Brief Search Engine History
Web did not have good search engines for a long time. The first search engines did not even analyze page copy; they only looked at titles and had no ranking criteria. As the convenience and commercial potential of search engine became more obvious, more advanced systems were developed.
Excite was the first serious commercial search engine. It was developed in Stanford and was purchased for $6.5 billion by @Home. In 2001 Excite and @Home went bankrupt and InfoSpace bought Excite for $10 million.
At the time the first search engines were rolling out, web directories were still strong competitors, primarily because of poor search results, and later on, because of spam and abuse.
Meta Tags
Meta tags were designed to help search engines sort web pages. Pages included keywords in meta tags telling search engines about the contents of each page. For a short time meta tags worked and helped search engines serve relevant results, but over time marketers learned they could easily rank by stuffing those tags with keywords.
As a result, search engine optimization in those days became about cramming "loans, loans, loans, loans, loans" into the meta tag. Search engines got spammed beyond being of any use, and many faced an exodus of users as a result.
Next: Important Search Engines >>
More Search Engine News Articles
More By Ivan Strouchliak
/ 3 
