Results 1 to 7 of 7

Thread: What are Spiders, Crawlers and Robots?

  1. #1
    Join Date
    Sep 2011
    Posts
    146

    Default What are Spiders, Crawlers and Robots?

    Search engines work by sending out what are called spiders, crawlers, or robots. Essentially, they crawl through cyberspace looking for webpages. When they get to a page, they take a sort of picture of the page, indexing the information on it, and integrating it into their giant database of millions of websites. When you type a word into the search engines looking for websites with content on that topic, in a split second they go through billions of webpages that the spiders and robots have visited. Webpages are typically revisited every few weeks.
    ..........
    plc training in chennai | industrial automation training chennai

  2. #2
    Join Date
    Jul 2010
    Posts
    112

    Default

    A spider is nothing more than a computer program that follows certain links on the web and gathers information as it goes. For example, Googlebot will follow HREF or SRC tags to find pages and images that are associated with any given site. Because these crawlers are merely computer programs, they aren't always the smartest of creatures and may get caught in endless loops.

    Corner Sofas Dublin

  3. #3
    Join Date
    Apr 2012
    Posts
    80

    Default

    Hello everyone,

    Spiders - It is a kind of program that follows links throughout the internet, grab the contents from the database and add it to the search engine database.

    Crawlers - Crawlers are also a program that crawls or follows all the links on each web page

    Robots - Robots are an automated computer program that visits the website which are guided by the Search engine algorithm. They perform crawling and indexing of the website to rank them in a particular search engine.

    Thanks and Regards
    Richard Smith

  4. #4

    Default

    Spiders,which are automated programs that examine web and create search-able indexes.The spiders gather information about your website and create indexes for the visitors searching the web.
    Crawlers are the programs that search engine use to scan and analyze websites in order to determine their importance,and thus their ranking in the results of internet searches for certain keywords.Different web crawlers work in distinct ways,but all web crawlers have the mission of exploring different websites available on the internet and retrieving suitable information from them so that they can be indexed by the search engine approximately.
    Robot is a catch-all,or generic term for programs and automated scripts that 'crawl' through the web and collect data from websites and anything else from the internet that they can find.

  5. #5

    Default

    Spiders,which are automated programs that examine web and create search-able indexes.The spiders gather information about your website and create indexes for the visitors searching the web.
    Crawlers are the programs that search engine use to scan and analyze websites in order to determine their importance,and thus their ranking in the results of internet searches for certain keywords.Different web crawlers work in distinct ways,but all web crawlers have the mission of exploring different websites available on the internet and retrieving suitable information from them so that they can be indexed by the search engine approximately.
    Robot is a catch-all,or generic term for programs and automated scripts that 'crawl' through the web and collect data from websites and anything else from the internet that they can find.

  6. #6
    Join Date
    May 2012
    Posts
    30

    Default

    Spiders or Crawlers or Robots are nothing but a computer program that enables the search engines into crawl into the website to update the details of the website periodically.

  7. #7

    Default

    These are simple software program designed to crawl through various links over the internet.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •