Spiders or robots are software that the various search engines send out to index websites. They bring back information to the search engines’ databases to keep them as up-to-date and relevant as possible. If you care about your standing in the search engine results pages (SERPs) – and who doesn’t? – you need to know how you can set up your site to make sure these programs can do their job smoothly. The articles in this section will tell you what to do, and just as importantly, what to avoid doing.
Search Engine Spiders Help 