Scraping Automotive Websites

What we need; Development, implementation & support of the Website and the Crawlers what we ask; fully commented code, open source solutions, documentation & support. General consideration; We want the software/site in English. A. Develop and implement 5 web crawlers to run continuously as a batch job in background updating the mysql db and tracking information changes from sites List; [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] The scraper should retrieve ALL the information about the listing and store it directly on our DB. The script should also be able to detect duplicate autos and avoid them. We prefer Python (Scrapy) scripts for the crawlers but this could be implemented in some other language. In the ideal scenario the script should never stop running. A second script for each crawler should be also developed and implemented in order to clean the DB and remove un-published listings . You will also have to create a [login to view URL] where a site admin may be able to see the scrapper progress, restart it, see logs, stop, etc 1st Requirements: Experience to support Crawlers on Amazon Web Services. Apache and MySQL DB 2nd Requirement: Experience scraping/crawling websites using Python 3es Requirement: Must have strong PHP and MYSQL DB background 3rd Requirement: Need some one who will be able to support Crawler DB and is interested in a part time position. Please let me know your hourly rate in PM If you are not able to support crawler on Amazon Web Services please don't bid.