A class library for crawling websites and taking actions when different urls are encountered. This can be used for: downloadng sites/images/movies, collecting pages for testing, computing statistics about websites, or anything else that involves
crawling the wev.
v1.0 was published on 8/14/2007.
This release includes only the class library, tests and documentation. In a soon-to-release version we will have a GUI for on-the-fly crawling not integrated into larger apps.