METHOD AND SYSTEM FOR CRAWLING THE WORLD WIDE WEB
Number of patents in Portfolio can not be more than 2000
United States of America Patent
Stats
-
N/A
Issued Date -
Nov 19, 2009
app pub date -
May 13, 2008
filing date -
May 13, 2008
priority date (Note) -
Abandoned
status (Latency Note)
Importance
Abstract
A method and system for crawling the World Wide Web is described. One embodiment avoids becoming bogged down by dynamically generated Uniform Resource Locators (URLs) pointing to Web pages having the same or substantially similar content (e.g., URLs generated by a “spam poison” Web site) by browsing automatically and systematically Web pages within a first domain of the World Wide Web, each Web page having its own content; determining that the content of a currently visited Web page is the same as that of a predetermined number of other Web pages that have already been visited; and ceasing to browse the first domain and instead browsing a second domain of the World Wide Web different from the first domain in response to determining that the content of the currently visited Web page is the same as that of a predetermined number of other Web pages that have already been visited.
First Claim
Family
International Classification(s)
Cited Art Landscape
Patent Citation Ranking
Recipient Email Address
Recipient Email Address
Comment
Recipient Email Address
Add to Portfolio(s)
To add this patent to one, or more, of your portfolios, simply click the add button.
This Patent is in these Portfolios:
Add to additional portfolios:
Last Refreshed On:
Changes done successfully