Feasability Study Of Webscraping and Price Comparison Website

This project received 8 bids from talented freelancers with an average bid price of $179 USD.

Get free quotes for a project like this
Employer working
Project Budget
$30 - $250 USD
Total Bids
Project Description

* DO NOT POST AUTOMATICALLY ON THIS PROJECT * All replies should be carefully considered.

I want to create a site comparing two competing websites and their online shopping prices. Each shopping system has approximately 20 000 items and uses different technologies for their cart systems. One is quite advanced and uses ajax, the other is from the dark ages and uses frames. I am wanting to create a weekly comparison of these 20,000 items.

This involves:

1. Weekly automated webscraping of existing items without getting banned
(i.e. the updates may need to be batch jobs to prevent being banned from the sites)

2. Identification of new items added to each of the sites.

3. The creation of a shopping cart system that will create two shopping carts, effectively allowing the customer to compare prices at the end and
decide which provider to go with.

4. Ability to create reports comparing goods across a category or brand.

These requirements are simplified. Further details can be provided upon request for suitable / non-automated replies. What I am looking for is someone who can explain a method of doing this.

The deliverable will be to produce a diagram (using BPMN and / or UML) and a short description on how this would work, including the key steps involved.

I have a background in IT and Business Analysis so this should be quite simple to communicate to me. I have already had a look at the source code and would be willing to get up to speed with the required technologies. I already have an idea of how this may work, though have limited exposure to Web Scaping.

Given that this is a feasibility study, I'm not wanting to spend a lot on it.

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online