I offer to provide the software that meets your needs. It will be a solution in Java, built on top of my own already developed, demonstrable web-scraper tool (OmanaWIM Tool). The web-scraper tool is designed for complex web-scraping, is technically sophisticated and can scrape deep and wide - will prove very cost-effective when large number of sites are involved.
The scraper tool accepts input in xml about website and information to be scraped .
It can do log-in, process JavaScript / AJAX call results, chase multi-level links, post search-forms and handle pagination; can accept / process response in XML; can download images and files; is multi-threaded in a configurable way; can use proxies; supports user-specifiable filters; scraped info can be delivered in JSON or XML / posted to database or Excel/CSV.
Deliverables:
1. Perpetual Non-exclusive non-transferable node-bound Use Licence for the OmanaWIM Tool with executable Java Application for scraping the multiple web-sites.
2. Custom Java classes for continuous run and for populating database.
3. Input-Xml-Schema,
4. Input-xml-files for 3 sites. For more sites, extra @ $30./site.
5. Installation Guide
ABOUT ME:
1.I am a full-time freelancer, with 15+ years of rich experience in software development.
2. I have expertise in:
A. web application architecture,
B. design and model development(OOAD and UML) including design patterns,
C. Core & Enterprise Java,
D. database development,
E. XML-Schema and
F. NLP.