Search Engine Spiders
  Home arrow Search Engine Spiders arrow Page 2 - Spider Guts
SEO Chat Forums  
Choosing Keywords  
Google Optimization  
Link Trading  
MSN Optimization  
Search Engine News  
Search Engine Spiders  
Search Optimization  
Web Directories  
Website Marketing  
Website Promotion  
Website Submission  
Yahoo Optimization  
SEO Tools
Adsense Calculator
AdSense Preview
Advanced Meta-Tags
Alexa Rank Tool
Check Server Headers
Class C Checker
Code to Text Ratio
CPM Calculator
Domain Age Check
Domain Typos
Future PageRank
Google Dance
Google Keywords
Google Search
Google Suggest
Google vs Yahoo
Indexed Pages
Keyword Cloud
Keyword Density
Keyword Difficulty
Keyword Optimizer
Keyword Position
Keyword Typos
Link Popularity
Link Price Calculator
Meta Analyzer
Meta Tag Generator
Multiple Link Popularity
Page Comparison
Page Size
PageRank Lookup
PageRank Search
Robots.txt Generator
ROI Calculator 
S.E. Comparison 
S.E. Keyword Position 
Site Link Analyzer 
Spider Simulator 
URL Redirect Check 
URL Rewriting 
Mobile Linux 
APP Generation ROI 
IBM® developerWorks 
SEO Weekly Newsletter
 
Developer Updates  
Free Website Content 
 RSS  Articles
 RSS  Forums
 RSS  All Feeds
Write For Us Get Paid 
Request Media Kit
Contact Us 
Site Map 
Privacy Policy 
Support 
 USERNAME
 
 PASSWORD
 
 
  >>> SIGN UP!  
  Lost Password? 
SEARCH ENGINE SPIDERS

Spider Guts
By: David Fells
  • Search For More Articles!
  • Disclaimer
  • Author Terms
  • Rating: 4 stars4 stars4 stars4 stars4 stars / 66
    2004-10-12

    Table of Contents:
  • Spider Guts
  • The Core Elements for Page Relevance
  • What Hurts Ranking

  • Rate this Article: Poor Best 
      ADD THIS ARTICLE TO:
      Del.ici.ous Digg
      Blink Simpy
      Google Spurl
      Y! MyWeb Furl
    Email Me Similar Content When Posted
    Add Developer Shed Article Feed To Your Site
    Email Article To Friend
    Print Version Of Article
    PDF Version Of Article
     
     
    ADVERTISEMENT


    Spider Guts - The Core Elements for Page Relevance


    (Page 2 of 3 )

    There are a number of things that a spider expects to see when it looks at a web page, many of which are optional but still important in the big picture. The following is a list that describes the core elements considered by a typical search engine when calculating page relevance.

    1. Title Tag - The title tag should contain a title relevant to the page, not just "Home Page" or "Contact Us". The title should be used for up to five keywords.

    2. Headings - Search engines view <h> tags as terms of emphasis, meaning additional weight is given to terms that appear inside them. Keywords should appear in <h> tags.

    3. Bold - Also viewed as terms of emphasis, but with less weight than headings.

    4. Alt Text - Brief descriptive sentences should be used in image alt attributes. At least one keyword should appear in each alt attribute.

    5. Keyword Meta Tag - Some engines use the keyword meta tags directly, some use them as part of a validation process ensuring that the keywords closely match the page content. The latter is the more typical scenario for modern engines. Keywords should be chosen carefully and be specific to the page they appear on.

    6. Description Meta Tag - Most search engines use this tag in a similar fashion as the keyword tags. Each page should have a unique description. The description should contain a few keywords and briefly summarize the content that appears on the page with a high degree of accuracy.

    7. Keyword Placement - Terms that are higher up on a page are more heavily weighted.

    8. Keyword Proximity - Terms that are close together are probably related, and thus the site will show up in searches for those terms.

    9. Comment Tags - Some search engines use comment tags for content, particularly in graphics rich/text poor sites.

    10. Page Structure Validation - Proper coding is likely to be of better overall quality, and thus rewarded.

    11. Traffic/Visitors - Search engines keep track of how many people follow their links. The more a link is followed for a given search, the more relevant the link is assumed to be.

    12. Link Popularity - Also known as PageRank, this is a measure of how many web pages on the Internet link to your site and the relevance of those pages to the page they are linking to. The popularity of the linking site is also evaluated.

    13. Anchor Text for Inbound Links - This is a measure of the relevance of the anchor text from the referring site.

    14. Page Last Modified - Newer content is regarded as "fresh" and is treated as more relevant.

    15. Page Size - Engines tend to weigh content at the start of a document more than content further down. If a page is too long, typically more than 50k in markup only, then it should be broken up into multiple pages.

    16. Keywords in URL - URLs are considered important by engines. Use of hyphens rather than underscores in filenames and using keywords in filenames and directories improves a pages potential relevance.

    These elements are all poured into an algorithm by the search engine that produces a very specific result: a relevance score for a page based on a given set of keywords. Evaluating page relevance is a constant reciprocal process that involves crawling around all pages indexed by a particular engine and evaluating the relevance of their content and the relevance of references to that content. The items listed above are things search engines expect to find in a page as well as factors that are not necessarily expected, but are considered if available (such as inbound links).

    More Search Engine Spiders Articles
    More By David Fells


       · Very good wrap up of the basics of SEO. These are very good optimization techniques...
       · I read in a few places that comments were not ignored if a site contained very...
       · Do you think each page of the site should have unique keywords and description of...
       · While the article itself was interesting and full of decent information, the title...
       · interesting
       · I am new to all of this and enjoyed your article a great deal. A have a somewhat...
     

    SEARCH ENGINE SPIDERS ARTICLES

    - The Yahoo SLURP Crawler
    - How Search Engines Work (and Sometimes Don’t)
    - Spider Guts
    - Score One for the Spiders?
    - Protect Against Invaders by SPAM-Proofing Yo...
    - ROBOTS.TXT Primer
    - Designing Websites For Humans In A World Of...





    © 2003-2009 by Developer Shed. All rights reserved. DS Cluster 1 Hosted by Hostway
    Stay green...Green IT