Invention Grant
US07899807B2 System and method for crawl ordering by search impact 有权
通过搜索影响来抓取排序的系统和方法

System and method for crawl ordering by search impact
Abstract:
An improved system and method for crawl ordering of a web crawler by impact upon search results of a search engine is provided. Content-independent features of uncrawled web pages may be obtained, and the impact of uncrawled web pages may be estimated for queries of a workload using the content-independent features. The impact of uncrawled web pages may be estimated for queries by computing an expected impact score for uncrawled web pages that match needy queries. Query sketches may be created for a subset of the queries by computing an expected impact score for crawled web pages and uncrawled web pages matching the queries. Web pages may then be selected to fetch using a combined query-based estimate and query-independent estimate of the impact of fetching the web pages on search query results.
Public/Granted literature
Information query
Patent Agency Ranking
0/0