A web crawler is an automated program that traverses web sites by following the links present on web pages. Crawlers are widely used by web search engines to index all the pages on a site by following the links from page to page. Web crawler programs are also known as spiders, robot, ants, automatic indexers, bots, and worms.
More Website and Internet
More Website and Internet
No comments:
Post a Comment