Build 2 web scraping tools and populate MySQL database!


Stagecoach Software Pty Ltd requires a freelancer to create two automated web scraping tools and a MySQL database suitable to serve as a back end to a forthcoming website to be built on Ruby on Rails.

The first tool needs to run on an automated schedule, initially at a set time every week. The tool is required to scrape a single large web site listing thousands of unique products and then populate a MySQL database with details of the products and prices sought. The details to be scraped appear in a consistent format on the site. The first scrape is required to obtain details for all products whilst subsequent scrapes are required to track new products listed or changes of price on existing products. The freelancer must populate the DB using the scraper as part of the project.

The second tool is required to automatically download and scrape 2 to 4 pdf consistently formatted text files published every week on a different website that list the realized prices for a significant proportion of the products listed in the first website. The second tool is required to populate a further table in the MySQL database with these products and price results.

The MySQL database must then automatically combine the data from the two scraping tools and calculate the difference between the realized price and the initial listed price(s). The DB must be loaded to a host site nominated by me. The scraper tools must be able to be easily activated from the host site and full operating instructions must be provided.

Full details of the sites to be scraped and the required DB columns will be supplied to the successful bidder.

This project will be followed with the creation of a RoR website that will present graphs showing aggregate and individual data obtained from the database and allow user searches. Preference will be given to bids that make that better facilitate this second stage.

Skills: MySQL, Ruby on Rails, Web Scraping

See more: build populate mysql database, web searches database, web scraping site freelancer, web scraping price, web scraping part time, web scraping c#, web host site, web data scraping freelancer, web creation freelancer, web build freelancer, tools for freelancer, sites using ruby on rails, serve as a freelancer, scraping tools web, scraping data from web database, ruby price, ruby on rails stage, ruby on rails sites, ruby on rails host, ruby on rails download, ruby on rails database, ok tools download, mysql database software, large format price list, host ruby on rails

About the Employer:
( 0 reviews ) hue, Vietnam

Project ID: #4975494

18 freelancers are bidding on average $1309 for this job


Dear Client, I can help in your project. We have already experience of working on similar projects. Please see below to get idea of our experience: Amazon/Ebay Bots: More

$773 USD in 10 days
(88 Reviews)

Could be done. More details required would be great.

$2000 USD in 10 days
(38 Reviews)

Hi.. Expert web scraper/Data Minor here. Interested in your project. I assure you 100% accurate and good quality work. Regards

$842 USD in 3 days
(44 Reviews)

We offer ; Programming (ROR, front-end) and agile development - Custom module development - Troubleshooting: ROR, theme, extensions, customizations - ROR control panel assistance - Third-party extension installati More

$1020 USD in 21 days
(14 Reviews)

Here is our live sites, We did alot work ing wordpress, joomla, drupal, Ruby on rails, Magento, opencart etc. We work in milestones and pay after seeing the result. No upfront. See here live sites. www.tribwa More

$3092 USD in 20 days
(26 Reviews)

Hi, Ready to start your work. Eagerly awaiting for your positive reply. I've very good experience in scraping and use Java as the language for scraping. The scraping tools can be scheduled to run periodically. I under More

$750 USD in 3 days
(26 Reviews)

Hi, Please look into my following profiles with multiple of companies, https://www.odes More

$1500 USD in 50 days
(12 Reviews)

Hello, Thanks, for giving us a chance to bid on your project. Please check private message box for more details. We are WebGloabalIt is an (ISO 9001:2008) (ISO 27001:2005) certified company who have throught 5 More

$1159 USD in 21 days
(5 Reviews)

Hello, I'm an experienced Web developer (6+ years, Zend PHP Engineer certification, Oracle MYSQL developer certification). For the last 2 years I've worked Ruby and Ruby on Rails. I have experience writing content More

$1333 USD in 14 days
(5 Reviews)

Hello Sir/Madam I have 3+ years of experience in web scraping. I have scraped more than 900 websites. I can do this for you. I have No.1 Software for web scraping and data extraction from the internet. The origi More

$750 USD in 1 day
(4 Reviews)

Good evening. I am computer enginner with most web development made in RoR. I need more details regarding the places to scrap info from, but seems I am able and ready to do the job. Thanks in advance. Best regards,

$1111 USD in 20 days
(1 Review)

I've been writing scraper code since 1998, and on a project I'm a part owner in for the last four years, using a variety of languages and technologies: C#, VB, ColdFusion, and Ruby. This covers a variety of tools, incl More

$1111 USD in 5 days
(1 Review)

Hi doangthehieu352, I'm Ian, a ruby developer. I'm interested in this project and would like to have opportunity to work on this project. I have 3 years experience working as a ruby/ror developer, currently work More

$800 USD in 5 days
(0 Reviews)

Hi, Yes, let's get it done. I am interested to join you and make an immediate impact. I am ready to start immediately and I am sure that i can work on your requirements with an confident manner. I have strong More

$1250 USD in 20 days
(0 Reviews)

Hi, WE HAVE ALREADY DONE this job and we have a demo that we want show you. Please reply in private.

$1250 USD in 1 day
(0 Reviews)

I offer to provide the software that meets your needs. It will be a solution in Java, built on top of my own already developed, demonstrable web-scraper tool (OmanaWIM Tool). The web-scraper tool is designed for comple More

$1250 USD in 3 days
(0 Reviews)

We, Veltrod Software Services ( are the fastest and largest growing provider of IT Software services for over 25% of the F100 and 100’s of Start-ups and venture funded firms. We are specialized on the be More

$1443 USD in 25 days
(0 Reviews)

Hello, i am a exp web developer and have a team of excellent developers in NYC office. I can create your 2 web tools for scraping data from different websites for reasonable rates. Give me a chance to start th More

$1473 USD in 3 days
(0 Reviews)

Hello We have just created a scraper in php/mysql to scrap data from 35000 pages in daily and monthly mode. You can see that project in our profile. Please have a look at that project and if you show interest we can d More

$1666 USD in 15 days
(0 Reviews)