Closed

200+ site screen scraper for single product comparison

This project was awarded to gbarco for $130 USD.

Get free quotes for a project like this
Employer working
Awarded to:
Project Budget
$30 - $250 USD
Total Bids
5
Project Description

I want the framework for a multi site scraper written so I can get the price, description, link and availability (in-stock/out of stock) of a single product that is sold on 200+ sites. Preferably, it would use CSS/jquery selectors to find the text to scrape.

I need to be able to put the links in to the pages to be scraped in myself and also the selectors. I would like the scraper to run on a schedule and output the data in SQL. I need the format in columns, with as many rows as there are pages to be scraped. The columns would be "id, description, price and status". The main thing is that I can select what selectors text from each site goes into what column and that it can be a different selector on every row. I understand manually going through 200+ sites to find on where their price information for 1 product is kept will be a lot of work, but it will be me doing it. I already started it using pjscrape, but I can't get all the data from multiple sites organized.

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online