Search Engine Spiders
  Home arrow Search Engine Spiders arrow Page 4 - How Search Engines Work (and Sometimes...
SEO Chat Forums  
Choosing Keywords  
Google Optimization  
Link Trading  
MSN Optimization  
Search Engine News  
Search Engine Spiders  
Search Optimization  
Web Directories  
Website Marketing  
Website Promotion  
Website Submission  
Yahoo Optimization  
SEO Tools
Adsense Calculator
AdSense Preview
Advanced Meta-Tags
Alexa Rank Tool
Check Server Headers
Class C Checker
Code to Text Ratio
CPM Calculator
Domain Age Check
Domain Typos
Future PageRank
Google Dance
Google Keywords
Google Search
Google Suggest
Google vs Yahoo
Indexed Pages
Keyword Cloud
Keyword Density
Keyword Difficulty
Keyword Optimizer
Keyword Position
Keyword Typos
Link Popularity
Link Price Calculator
Meta Analyzer
Meta Tag Generator
Multiple Link Popularity
Page Comparison
Page Size
PageRank Lookup
PageRank Search
Robots.txt Generator
ROI Calculator 
S.E. Comparison 
S.E. Keyword Position 
Site Link Analyzer 
Spider Simulator 
URL Redirect Check 
URL Rewriting 
Mobile Linux 
APP Generation ROI 
IBM® developerWorks 
SEO Weekly Newsletter
 
Developer Updates  
Free Website Content 
 RSS  Articles
 RSS  Forums
 RSS  All Feeds
Write For Us Get Paid 
Request Media Kit
Contact Us 
Site Map 
Privacy Policy 
Support 
 USERNAME
 
 PASSWORD
 
 
  >>> SIGN UP!  
  Lost Password? 
SEARCH ENGINE SPIDERS

How Search Engines Work (and Sometimes Don’t)
By: Terri Wells
  • Search For More Articles!
  • Disclaimer
  • Author Terms
  • Rating: 4 stars4 stars4 stars4 stars4 stars / 49
    2005-12-26

    Table of Contents:
  • How Search Engines Work (and Sometimes Don’t)
  • Stumbling Instead of Crawling
  • More Stumbling Blocks
  • What Do Spiders See in a Hyperlink?

  • Rate this Article: Poor Best 
      ADD THIS ARTICLE TO:
      Del.ici.ous Digg
      Blink Simpy
      Google Spurl
      Y! MyWeb Furl
    Email Me Similar Content When Posted
    Add Developer Shed Article Feed To Your Site
    Email Article To Friend
    Print Version Of Article
    PDF Version Of Article
     
     
    ADVERTISEMENT


    How Search Engines Work (and Sometimes Don’t) - What Do Spiders See in a Hyperlink?


    (Page 4 of 4 )

    I’ll assume that you are all reasonably familiar with HTML. If you have ever looked at the source code for an HTML page, you probably noticed text like this wherever a hyperlink appeared:

    When a web browser reads this, it knows that the text “SEO Chat” should be hyperlinked to the web page http://www.seochat.com. Incidentally, “SEO Chat” in this case is the “anchor text” of the link. When a spider reads this text, it thinks, “Okay, the page http://www.seochat.com is relevant to the text on this page, and very relevant to the term `SEO Chat.’”

    Let’s get a little more complicated.

    Now what? The anchor text hasn’t changed, so the link will still look the same when the web browser displays it. But a spider will think, “Okay, not only is this page relevant to the term `SEO Chat,’ it is also relevant to the phrase `Great Site for SEO Info.’ And hey, there’s a relationship between the page I’m crawling now and this hyperlink! It says that this link doesn’t count as a ‘vote’ for the page being linked to. Okay, so it won’t add to the page rank.”

    That last point, about the link not counting as a vote for the page being linked to, is what the rel="nofollow" tag does. This tag evolved to address the problem of people submitting linked comments to blogs that said things like "Visit my pharmaceuticals site!" That kind of comment is an attempt by the commenter to raise his own website's position in the search engine rankings. It's called comment spam, by the way; most major search engines don't like comment spam because it skews their results, making them less relevant. As you may have guessed, then, the “nofollow” tag in the “rel” attribute is specifically for search engines; it really isn't there to be noticed by anyone else. Yahoo!, MSN, and Google recognize it, but AskJeeves does not support nofollow; its crawler simply ignores the nofollow tag.

    In some cases, a link may be assigned to an image. The hyperlink would then include the name of the image, and might include some alternate text in an “alt” attribute, which can be helpful for voice-based browsers used by the blind. It also helps spiders, because it gives them another clue for what the page is about.

    Hyperlinks may take other forms on the web, but by and large those forms do not pass ranking or spidering value. In general, the closer a link is to the classic <a href=”URL”>text</a>, the easier it is for a spider to follow a link, and vice versa.


    DISCLAIMER: The content provided in this article is not warranted or guaranteed by Developer Shed, Inc. The content provided is intended for entertainment and/or educational purposes in order to introduce to the reader key ideas, concepts, and/or product reviews. As such it is incumbent upon the reader to employ real-world tactics for security and implementation of best practices. We are not liable for any negative consequences that may result from implementing any information covered in our articles or tutorials. If this is a hardware review, it is not recommended to open and/or modify your hardware.

       · I hope you found this article informative and entertaining. I welcome your...
       · No Follow tag, you address this, but why do it? You address better site maps, what...
       · Actually, I did explain what the nofollow tag does, but not very explicitly; I've...
       · Hello there, I found your article very interesting.However, I am still no...
       · Just from that description, I really can't tell -- but if you check out our forums...
       · I 've the same problem with my website Fitness-Boat. can anyone suggest few step...
       · This may be a bit outdated from when the message was first posted, but you can get...
       · thanks terri,i followed your article and bang my page rank shot up.my website is...
       · Thanks Frank, I'm really delighted to hear about your success! I'm glad you found my...
       · hi terri,i would like to publish your article on my web site.
       · Thanks for asking, but I can't give you permission to republish this article on your...
       · Your article is very well done. However, one thing I do not understand;since there...
       · Okay, first of all, nobody is bribing the search engines to get a higher ranking....
     

    SEARCH ENGINE SPIDERS ARTICLES

    - The Yahoo SLURP Crawler
    - How Search Engines Work (and Sometimes Don’t)
    - Spider Guts
    - Score One for the Spiders?
    - Protect Against Invaders by SPAM-Proofing Yo...
    - ROBOTS.TXT Primer
    - Designing Websites For Humans In A World Of...





    © 2003-2010 by Developer Shed. All rights reserved. DS Cluster 8 Hosted by Hostway
    For more Enterprise Application Development news, visit eWeek