Feasability Study Of Webscraping and Price Comparison Website

  • Status Closed
  • Budget $30 - $250 USD
  • Total Bids 8

Project Description

* DO NOT POST AUTOMATICALLY ON THIS PROJECT * All replies should be carefully considered.

I want to create a site comparing two competing websites and their online shopping prices. Each shopping system has approximately 20 000 items and uses different technologies for their cart systems. One is quite advanced and uses ajax, the other is from the dark ages and uses frames. I am wanting to create a weekly comparison of these 20,000 items.

This involves:

1. Weekly automated webscraping of existing items without getting banned

(i.e. the updates may need to be batch jobs to prevent being banned from the sites)

2. Identification of new items added to each of the sites.

3. The creation of a shopping cart system that will create two shopping carts, effectively allowing the customer to compare prices at the end and

decide which provider to go with.

4. Ability to create reports comparing goods across a category or brand.

These requirements are simplified. Further details can be provided upon request for suitable / non-automated replies. What I am looking for is someone who can explain a method of doing this.

The deliverable will be to produce a diagram (using BPMN and / or UML) and a short description on how this would work, including the key steps involved.

I have a background in IT and Business Analysis so this should be quite simple to communicate to me. I have already had a look at the source code and would be willing to get up to speed with the required technologies. I already have an idea of how this may work, though have limited exposure to Web Scaping.

Given that this is a feasibility study, I'm not wanting to spend a lot on it.

Get free quotes for a project like this

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online