Search Directories For the Invisible Web
(Page 1 of 4 )
Google indexes billions of Web pages, but there are many more billions of Web pages and websites that the major search engines cannot reach. In fact, by some estimates, this "Invisible Web" is 500 times the size of the visible one. Fortunately for searchers, there are ways to access this treasure trove. This article explains how search directories work, and what can be found by using some of the major ones.
In a recent article I discussed the existence of a vast number of documents that, due to a variety of reasons, cannot be retrieved using the major search engines. But the so-called Invisible Web exists, and many treasures are hidden from easy access in various ways. Some sit behind the thick “firewalls” of sites that require registration to be searched. Others have complex database structures that search engine crawlers cannot technically deal with. Still others are hosted on a “black list” domain, or are simply skipped because search engines cannot index every single page on the Web the moment it appears.
Still, these pages exist, and their number significantly exceeds the number of pages that are accessible via the major search engines. According to rough estimates, the Invisible Web is 500 times bigger than what is served by search engines. And what is more important, often the information you search for cannot be found (or at least not easily) using the major search engines, while at the same time it is just a couple of clicks away from you. And if you do not know how to search for it, you might never find it.
There are special tools for searching this vast ocean of information. True, they are not as convenient as searching with Google, and they take much more time and effort, but when you need very specific information this might be the only way to get it. The two major tools for searching the Invisible Net are the specialized search engines and the search (known also as subject) directories. Additionally, using the Invisible Web databases and virtual reference libraries often can deliver you the information you need, but the two major tools are the search engines and the search directories for the Invisible Web.
Next: Search Engines vs. Directories >>
More Search Optimization Articles
More By Tsvetanka Stoyanova
