The Missing Link - Can You See What The Spider Will Do?
(Page 5 of 9 )
On the first visit, the spider starts on the homepage and finds sitemap page 1. The spider will then find every page that sitemap page 1 links to. I.E. The spider will find all other site map pages, plus all standard site pages that are linked to from site map page 1.
On the second visit (month 2) the spider can start on each and every sitemap page, as it found them all during its last visit. Remember the spider can start on any page it knows about, and will visit your site several times during a month. From the sitemap pages, the spider can find all other pages of the site by following the page links.
So, for even large 1000+ page web sites, the spider takes only 2 months to collect information about every page. That is not bad.
Submitting the Sitemap to the Search Engines
Something that can speed up the spidering of a new site that is not yet in the search engines is if you submit the sitemap page 1 to the search engine manually. Then the spider can start on sitemap page 1 and follow links to every single page in the same visit (the site is 2 levels deep from sitemap page 1).
Alternatively, if you have a page already in Google – perhaps on a different site, just link to the sitemap from it. This can get you included very quickly.
1000 Page Site Map? You Have to be Kidding!
Unfortunately, sitemaps do take time to build. On a 1000-page site, this could take many hours or even days to do properly. Even relatively small sitemaps can take several hours to build and check. There is, however, a software solution that can cut those hours & days into minutes. Before we look at it, we need to consider what makes a good sitemap page.
Anatomy of a Good Sitemap Page
A sitemap page is a list of links to other pages on your site. If you just provide a list of links, the page will look very skeletal and of little value to your visitors. Yes, a spider can find your pages, but why not design a sitemap page that adds value to your site for the visitors too? With that in mind, each link should probably have a description of what the target page is about for each link. This will ensure that human visitors can use your sitemap as a navigation system.
Next: Two More Examples >>
More Link Trading Articles
More By Andrew Williams
