Using Your Web Stats for SEO: Search Marketing Analysis from Web Stats - How to Use Web Stats Continued
(Page 4 of 6 )
You should also look for search engine spiders using sessions. Search engine spiders do not accept cookies, or private files on a user’s hard drive, so sessions are used for those who cannot or won’t allow cookies. If spiders are not prevented from using sessions, then your pages could be indexed with those session id’s attached to the end of your URL. For example, if your page to be indexed is:
http://recipe-website.com/c/chocolate-chip-cookies.php
and a session id is attached to it, making it look like:
http://recipe-website.com/c/chocolate-chip-cookies.php?cID=ab12345cde45f89i
then when it indexes the URL with the session, this will cause problems. If a spider has indexed a site with the session id, it will follow its databased link, only to encounter a 404 Page Not Found error when it cannot recreate that session. Your web stats will be able to alert you both with the pages that were crawled by the spider, and what errors it has encountered.
The solution to this problem would be to install a script that would not allow spiders to create session ids. You may need to view your shopping cart user’s manual for more information.
Even knowing the approximate times of day that the search engine spiders crawl your site can be helpful. For example, Google usually crawls our website at 11 pm central time. What I can determine from this statistic is not to upload new files or scripts that it will have trouble crawling during this period of time, and either do it before the potential crawl times, or wait until after. There is no way to know when a spider will crawl your site, but analyzing your web stats can help you be more prepared, and help you understand the crawling patterns of each search engine.
Next: Tips for Using Web Stats for SEO >>
More Website Marketing Articles
More By Jennifer Sullivan Cassidy
/ 9 
