* DO NOT POST AUTOMATICALLY ON THIS PROJECT * All replies should be carefully considered.
I want to create a site comparing two competing websites and their online shopping prices. Each shopping system has approximately 20 000 items and uses different technologies for their cart systems. One is quite advanced and uses ajax, the other is from the dark ages and uses frames. I am wanting to create a weekly comparison of these 20,000 items.
1. Weekly automated webscraping of existing items without getting banned
(i.e. the updates may need to be batch jobs to prevent being banned from the sites)
2. Identification of new items added to each of the sites.
3. The creation of a shopping cart system that will create two shopping carts, effectively allowing the customer to compare prices at the end and
decide which provider to go with.
4. Ability to create reports comparing goods across a category or brand.
These requirements are simplified. Further details can be provided upon request for suitable / non-automated replies. What I am looking for is someone who can explain a method of doing this.
The deliverable will be to produce a diagram (using BPMN and / or UML) and a short description on how this would work, including the key steps involved.
I have a background in IT and Business Analysis so this should be quite simple to communicate to me. I have already had a look at the source code and would be willing to get up to speed with the required technologies. I already have an idea of how this may work, though have limited exposure to Web Scaping.
Given that this is a feasibility study, I'm not wanting to spend a lot on it.
8 freelancers are bidding on average $179 for this job
We are a Web design and development company that focuses on regular communication with our clients and timely delivery for all our projects. Some of the technologies/area that we focus include: