资 源 简 介
This project is to create a Web Crawler which uses a network of computer to improve computing operations.
The application is a sofware agent that visits a list of URLs and identifies all the hyperlinks in the page and adds them to the list of URLs to visit.
The main objectifs are to :
- Analyze a large volume of data.
- Visit lot URLs in few time.
- Extract content subject (text mining) in each page.
- Create a graph of a social network from the Internet.
