This is primarily a web scraping application running on Linux/Apache/MySQL/PHP (LAMP) framework. Must use a batch framework to allow parallel processing of module execution. Must implement or extend a scraping framework which will allow the information to be scraped and stored in the database. Must allow modules to be easily added to the application and to the scraping tests.
The URLs which are scanned may return a 200 (OK) or one of several error responses. We'll only want to scrape data from the successful requests.
Must be able to create the Schema for the database.
Must be able to work well with me (good communication and willing to ask questions rather than make assumptions.) Must be able to complete the project by mid-June. Must be able to make this production-ready for use by non-technical users. Cannot cut corners or take short-cuts.
Must be willing to sign an NDA upon accepting the project.
22 freelancers are bidding on average $1350 for this job
Highly experienced web scraper and website developer. Have developed hundreds of web scrapers and website in the past 7 years. More info in the pm. Cheers, SayNo2Bugs
Seasoned web scraper. I worked on many similar projects, I have big experience in data mining projects. I can finish this task in short time, with the best quality.
Hi! The project you described can be either straightforward or quite challenging depending on your detailed requirements. It won't be a problem for me either way. I have also sent a PM to you.