 |
- Title
- Designing Efficient Topic-Driven Web Crawlers(TR02-15.ps)
- Author(s)
- Yiqiao Wang, Eleni Stroulia
- Technical Report
- TR02-15, Jul 2002
- Keywords
- No keywords provided
- Abstract
- Crawlers are essential to web search engines for retrieving high quality web pages automatically and
efficiently based on developer defined notions of importance and quality. Due to rapid growth of World-Wide Web and limited resources available to crawlers, developing good crawling strategies and evaluating them are still big challenges. In this paper, we do a comprehensive study of existing and proposed crawling strategies done by other research works. We have developed a topic-driven crawler that uses combinations of two different strategies in evaluating page importance during the crawl.
|
|
|
