Wednesday, September 9, 2009

WebCrawling software

WebCrawler software is a computer program used to browse the internet in a systematic, mechanical way. Many search engines such as Google uses this software as a means to provide current data. The WebCrawling software is used to create a copy of sites visited on the internet and processed by the search engine. Then it will index the pages that have down loaded for faster searches. A WebCrawler is also used to perform automated maintenance tasks on websites. It will collect information such as email addresses from web pages. WebCrawler software will try to download pages that are related to each other.

No comments:

Post a Comment