Spider

Mention a spider and the first thing you probably think of is those crawling, 8 legged, terrors that you would not come within 20 metres of. However an Internet spider is a bot that any webmaster is glad to have crawl right over their website! To be more precise spiders are used by search engines to crawl your page. This point may seem trival at first, but understanding how spiders crawl and index your pages, can be invaluable in improving your search engine standing.

There are different kinds of spiders that search engines use; deep, shallow and verification are the three main types.

A Deep spider is one that crawls through all levels of your web site, doesn't matter how deep it is. Altavista is an example of a search engine using deep spider. This type of spider, will study your site and bring the related information back to the search engine to store in the database, the information they are looking are like title, meta tag, relevance and what ever else search engines deem important for indexing. It also serves the secondary functions of verifying links, and remove any dead links in the main database.

A Shallow spider, is one that crawls the first level, or a few levels after that, an example of search engine using a shallow spider is Excite. It carries a similar process as the deep spider in terms of information gathered, but it stops at a pre-definded depth. So if you page is in 3 or 4 level deep with a website, there is a good chance, this type of spider will not crawl it.

A Verification spider, is a spider that is sent out to the site to verify that the URL actually exists. It does not bring back any information or particular about the web site. This is mostly used by the directory type of search engine such as Yahoo. This type of spider does not care what sort of title or what sort of content you have, they just come and check whether the site is active or not, then report back.

Above is the spider family and what they do. Even if two different engines use a shallow spider, they can differ again in terms how they collect and organize the data. Each engine has their own ranking algorithm based on what they think is important. Commonly they all favour content rich and theme based websites. This is the new direction search engine has move towards in order to return a better search result.

Generally there are a couple things the spider will look for; title, meta tags, texts in the page related to the keyword, checking whether you are spamming or not, and such. If you write whole page of nothing but the keyword, the spider is smart enough to pick that up and remove your listing, or worse, ban the domain. So a well designed page with rich content is often the best way to go.

For more information on web promotion via search engines, please visit the Dynamic Submission Homepage.

Act now and submit your web sites to hundreds of search engines by downloading the Dynamic Submission 2000.

Also check our related article, web spider...

Meta Tag Generator, an excellent read!

CLIENT TESTIMONIALS

• 01

  Stop! Don't spend another dollar of your hard earned money, UNTIL you've purchased Dynamic Submission! Let me tell you folks, this software rocks! It's so EASY to use, and the interface is well designed and not cumbersome. When I had all the information regarding my site entered into the software, and pressed submit, I had my doubts! I was expecting timeouts, for the software to stall, for a lot of the submissions to fail, BUT, this software just fires away, submitting my site to ALL those engines, and it worked flawlessly! I'm so impressed, I'll never use another submission service again! In fact, I plan on upgrading soon.
  
  Oh yeah, 1 more thing. The customer support is friendly and quick! You've got nothing to loose with Apex Pacific, buy your copy of Dynamic Submission today!

  Jeffrey Rookard , Webmaster, Tetra Hosting.

• 02

  Thank you so much for all of your help. It worked! You have been absolutely fabulous at helping me get this software going. I cannot say enough as to how much I appreciate your promptness and helpfulness with answering all of my questions! If only all tech support for online assistance would follow your suit... Thanks Again!

  Nikki

• 03

  On another note. I have tried a few mailing programs. And this one so far is the best. It is very simple to use, very quick to transmnit the messages, and so far has been doing a good job of pruning my lists as bounces and removal requests happen. Great job.

  Robert Johnson, Dreamkey Design

Click here for more testimonials

FEATURED PRODUCTS