Search Engine Spiders
  Home arrow Search Engine Spiders arrow Page 3 - Score One for the Spiders?
SEO Chat Forums  
Choosing Keywords  
Google Optimization  
Link Trading  
MSN Optimization  
Search Engine News  
Search Engine Spiders  
Search Optimization  
Web Directories  
Website Marketing  
Website Promotion  
Website Submission  
Yahoo Optimization  
SEO Tools
Adsense Calculator
AdSense Preview
Advanced Meta-Tags
Alexa Rank Tool
Check Server Headers
Class C Checker
Code to Text Ratio
CPM Calculator
Domain Age Check
Domain Typos
Future PageRank
Google Dance
Google Keywords
Google Search
Google Suggest
Google vs Yahoo
Indexed Pages
Keyword Cloud
Keyword Density
Keyword Difficulty
Keyword Optimizer
Keyword Position
Keyword Typos
Link Popularity
Link Price Calculator
Meta Analyzer
Meta Tag Generator
Multiple Link Popularity
Page Comparison
Page Size
PageRank Lookup
PageRank Search
Robots.txt Generator
ROI Calculator 
S.E. Comparison 
S.E. Keyword Position 
Site Link Analyzer 
Spider Simulator 
URL Redirect Check 
URL Rewriting 
Mobile Linux 
APP Generation ROI 
IBM® developerWorks 
SEO Weekly Newsletter
 
Developer Updates  
Free Website Content 
 RSS  Articles
 RSS  Forums
 RSS  All Feeds
Write For Us Get Paid 
Request Media Kit
Contact Us 
Site Map 
Privacy Policy 
Support 
 USERNAME
 
 PASSWORD
 
 
  >>> SIGN UP!  
  Lost Password? 
SEARCH ENGINE SPIDERS

Score One for the Spiders?
By: Terri Wells
  • Search For More Articles!
  • Disclaimer
  • Author Terms
  • Rating: 3 stars3 stars3 stars3 stars3 stars / 14
    2004-05-10

    Table of Contents:
  • Score One for the Spiders?
  • Copyright Infringement?
  • Spiders Gone Wild

  • Rate this Article: Poor Best 
      ADD THIS ARTICLE TO:
      Del.ici.ous Digg
      Blink Simpy
      Google Spurl
      Y! MyWeb Furl
    Email Me Similar Content When Posted
    Add Developer Shed Article Feed To Your Site
    Email Article To Friend
    Print Version Of Article
    PDF Version Of Article
     
     
    ADVERTISEMENT


    Score One for the Spiders? - Spiders Gone Wild


    (Page 3 of 3 )

    I imagine you can all see how this has relevance for spiders and those who aggregate information about specific products...or even auctions. So, does this mean that spiders can go out and disregard those software-based warnings from Web sites telling them that they're not welcome? Does this take eBay's winning of an injunction against Bidder's Edge and turn it on its head?

    You'd think it would. On the face of it, the two cases look similar. Bidder's Edge, an auction aggregation site, was sending spiders to eBay to collect information on eBay's auctions. BE would then aggregate eBay's auction information along with auction info from many other auction Web sites, making it a "one-stop shop" for that kind of information. eBay wanted them to stop -- and, in fact, was able to legally force them to stop.

    That is where the similarities between the two cases end. You see, eBay didn't claim that Bidder's Edge was infringing its copyright -- or, at least, the court didn't grant the injunction based on a claim of copyright infringement. Oh no. eBay claimed that Bidder's Edge was trespassing! Bidder's Edge admitted that it was sending 80,000 to 100,000 queries a day to eBay's Web site, and eBay argued that that was akin to sending 80,000 to 100,000 robots into a bricks-and-mortar business looking for prices and not buying anything. Well, the court was skeptical of that argument, but not of certain other arguments. Specifically, eBay was able to point out that Bidder's Edge's spiders were using up eBay's computer capacity, after eBay had told BE more than once that it was not welcome to use its spiders on eBay's Web site. At that time, it was only using less than two percent of eBay's capacity -- but the principle in this case wasn't the amount of capacity it was using up, but that it was using that capacity after eBay told it not to. A Web site might not actually be real estate in the same way that land is -- but there is no question that computers and servers are property that can be owned. And Bidder's Edge was using eBay's computers in ways that eBay specifically told it that it wasn't allowed to do. What would you do if you told someone not to use your computer in a particular way and they kept doing it? Uh-huh, that's what I thought. In eBay's case, they were granted an injunction against Bidder's Edge in late May 2000.

    So what does Nautical Solutions Marketing vs. Boats.com add to the sometimes-controversial issue of spiders? Well, first off, just because something was copied from a Web site -- even by a robot -- doesn't mean that the person or company doing it is committing copyright infringement. Especially if it's essentially factual information. On the other hand, that also doesn't mean that they're not in hot water. If you're a spider wrangler, make sure your spiders check to see whether they're welcome on the Web sites they visit -- and don't try to force the issue. It's a big Web out there; there's plenty of cyberspace for spiders to crawl without looking for trouble. 


    DISCLAIMER: The content provided in this article is not warranted or guaranteed by Developer Shed, Inc. The content provided is intended for entertainment and/or educational purposes in order to introduce to the reader key ideas, concepts, and/or product reviews. As such it is incumbent upon the reader to employ real-world tactics for security and implementation of best practices. We are not liable for any negative consequences that may result from implementing any information covered in our articles or tutorials. If this is a hardware review, it is not recommended to open and/or modify your hardware.

     

    SEARCH ENGINE SPIDERS ARTICLES

    - The Yahoo SLURP Crawler
    - How Search Engines Work (and Sometimes Don’t)
    - Spider Guts
    - Score One for the Spiders?
    - Protect Against Invaders by SPAM-Proofing Yo...
    - ROBOTS.TXT Primer
    - Designing Websites For Humans In A World Of...



     



    © 2003-2009 by Developer Shed. All rights reserved. DS Cluster 2 Hosted by Hostway
    For more Enterprise Application Development news, visit eWeek