Agent Name

October 23rd, 2008

Indicates the spider that is currently visiting a web page. Spiders are used by search engine to capture information from web sites and then to catalog them. Sited that have been indexed are termed as being spidered.

Related posts:

  1. Crawl
  2. Bait and Switch
  3. Filter Word
  4. Doorway page
  5. Dynamic Content
  6. Crawler
  7. Entry Page
  8. Deep submitting
  9. Cgi-bin
  10. Banned