网络爬虫分布在多台计算机

This project is to create a Web Crawler which uses a network of computer to improve computing operations. The application is a sofware agent that visits a list of URLs and identifies all the hyperlinks in the page and adds them to the list of URLs to visit. The main objectifs are to : - Analyze a large volume of data. - Visit lot URLs in few time. - Extract content subject (text mining) in each page. - Create a graph of a social network from the Internet.

mycrawler-source-0.2.1
.DS_Store
clean.bat
Components
Graph
Loopback capture.txt
MyCrawler.h
MyCrawler.pro
MyCrawlerClient
MyCrawlerServer
Protocol specifications.txt
QtGraph
Shared
TODO.txt