We need to develop a robust web spider that crawl web sites and extract data from them.
we need a Worker Threads that is constantly running, and the jobs must be executed separetly.
All results must be saved on a Database.
Winner must have already experience on this, and explain the architecture of the web spider.
We give more details in discussion