Need some work done? Post a Project Today
Attention Scriptlance users, Scriptlance has been acquired by Freelancer.com! Find out more
I'm looking for an expert who has experience with Nutch or Scrapy to help me set up a webcrawler to scan websites and webfiles and then update a database with the info.
Client-based user interface:
1. create/edit/remove rules
a. real-time webpage scan
b. real-time webpage + crawl scan (crawl means it follows links on the website to other pages, and then scans these pages, for X levels)
c. real-time file scan
d. real-time multiple/batch file scan
rules can be executed one time, continuously, or over a timeframe (every x minutes)
rules can be turned on/off
2. Display live status of the engine in an
3. write to database accroding to structure predefined in rules
Freelancer.com (formerly GetAFreelancer, Scriptlance and vWorker/Rentacoder) is the world's largest freelancing, outsourcing and crowdsourcing marketplace for small business. Hire freelancers to work in software, writing, data entry and design right through to engineering and the sciences, sales and marketing, and accounting & legal services.
Find freelance jobs and make money online! We have freelance coders, writers, programmers, designers, marketers and more. Getting the best web design, custom programming, professional writing or affordable marketing has never been easier!
© Copyright 2013 Freelancer Technology Pty Limited (ACN 142 189 759)
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)