Spiders. Those creepy, crawlies of data mining that scour the web for bits and pieces of information have a habit of getting into trouble. Just ask eBay and Boats.com, who recently had to resort to some legal bug spray in order to get rid of the little pests. Is your data scavenging in danger?
Read Article →

Benjamin Pfeiffer discusses how to SPAM-proof your website. He explains how to use Javascript and mod_rewrite to stop SPAMbots and Spybots from finding email addresses on your website. He also talks about how to find and set up the .htaccess file and gives examples of robots and how to block them.
Read Article →

There is often confusion as to the role and usage of the robots.txt file. I thought it would be a good idea to dispel some myths and highlight what robots.txt files are all about.
Read Article →