Project Description:
I'm looking for an expert who has experience with Nutch or Scrapy to help me set up a webcrawler to scan websites and webfiles and then update a database with the info.
Client-based user interface:
1. create/edit/remove rules
a. real-time webpage scan
b. real-time webpage + crawl scan (crawl means it follows links on the website to other pages, and then scans these pages, for X levels)
c. real-time file scan
d. real-time multiple/batch file scan
rules can be executed one time, continuously, or over a timeframe (every x minutes)
rules can be turned on/off
2. Display live status of the engine in an
3. write to database accroding to structure predefined in rules
Java, MySQL, PHP, Python, Web Scraping
Bids are hidden by the project creator.
Log in as the project creator or as one of the bidders to view bids.
You will not be able to bid on this project if you are not qualified in one of the job categories. To see your qualifications click here.
You will not be able to bid on this project if you are not qualified in one of the job categories. To see your qualifications click here.
users online now
2,849,171
freelancers
1,228,357
projects
