Build 2 web scraping tools and populate MySQL database!

CLOSED
Bids
19
Avg Bid (USD)
$1293
Project Budget (USD)
$750 - $1500

Project Description:
Stagecoach Software Pty Ltd requires a freelancer to create two automated web scraping tools and a MySQL database suitable to serve as a back end to a forthcoming website to be built on Ruby on Rails.

The first tool needs to run on an automated schedule, initially at a set time every week. The tool is required to scrape a single large web site listing thousands of unique products and then populate a MySQL database with details of the products and prices sought. The details to be scraped appear in a consistent format on the site. The first scrape is required to obtain details for all products whilst subsequent scrapes are required to track new products listed or changes of price on existing products. The freelancer must populate the DB using the scraper as part of the project.

The second tool is required to automatically download and scrape 2 to 4 pdf consistently formatted text files published every week on a different website that list the realized prices for a significant proportion of the products listed in the first website. The second tool is required to populate a further table in the MySQL database with these products and price results.

The MySQL database must then automatically combine the data from the two scraping tools and calculate the difference between the realized price and the initial listed price(s). The DB must be loaded to a host site nominated by me. The scraper tools must be able to be easily activated from the host site and full operating instructions must be provided.

Full details of the sites to be scraped and the required DB columns will be supplied to the successful bidder.

This project will be followed with the creation of a RoR website that will present graphs showing aggregate and individual data obtained from the database and allow user searches. Preference will be given to bids that make that better facilitate this second stage.

Skills required:
MySQL, Ruby on Rails, Web Scraping
About the employer:
Verified
Public Clarification Board
Bids are hidden by the project creator. Log in as the employer to view bids or to bid on this project.
You will not be able to bid on this project if you are not qualified in one of the job categories. To see your qualifications click here.


$ 773
in 10 days
Hire taro
$ 2000
in 10 days
$ 842
in 3 days
$ 1020
in 21 days
$ 3092
in 20 days
$ 750
in 3 days
$ 1500
in 50 days
$ 1159
in 21 days
Hire ltravenvw
$ 1333
in 14 days
$ 750
in 1 days