开源爬虫java文件

Filecrawler4j is an open-source Java crawler for crawling local files. Its light-weight, fast, and easy to setup. Strongly inspired by the popular web crawling project, Crawler4j, Filecrawler4j borrows many of the same configurations and shares a highly similar code syntax and structure to Crawler4j. If you"re familiar with Crawler4j, you"ll immediately pick up how to use Filecrawler4j. It only takes about 5 minutes to set up! UPDATE: Version 1.0 is released and ready for download! Sample Usage You will need to implement a controller where you will setup crawl configurations and decide where to store the data from the crawl. ``` public class Controller { public static void main(String args[]) throws Exception { String crawlStorageFolder = "data"; String pathToDirecto

filecrawler4j-1.0.jar
log4j.properties