Amazon web scraper, web scraping into Magento MySQL database

This project received 18 bids from talented freelancers with an average bid price of $601 USD.

Get free quotes for a project like this
Project Budget
$30 - $250 USD
Total Bids
Project Description

Hi all,

We would like to scrape Amazon products with photos and import them into our Magento website ([url removed, login to view]) by providing a URL to scrape from.

The solution has to be as simple as possible.

For example, we would like to scrape all 9000+ products from the URL:!493964%2Cn%3A541966%2Cn%3A1232597011&hidden-keywords=-B0089RPUHO+-B00DG0NUEE&ie=UTF8&qid=1385551805


1) You must create a Magento ADMIN menu for the scraper (we are not interested in running scripts via SSH etc)

2) Scraped products need to match our product columns (we will provide you with a CSV file so that you know what info needs to be scraped)

3) Your scraper will need to interact with our MySQL database and archive products that are no longer available (the scraped data needs to go directly into our database, we are not interested in creating XML or CSV files for import)

4) We must have the ability to scrape by brand, category, price range etc

5) We need to scrape all product info as directed, for example: SKU, title, availability, price, colour, description, short description and so on

6) You need to be able to employ current anti-scraping methods: proxies, throttling etc

7) The scraper needs to have an UPDATE function where we can scrape a URL and only update certain attributes (ie. price, availability) - that way we don't have to scrape all items in full again

So in a nutshell, we enter a URL in our Magento ADMIN and the scraper starts importing all products into our MySQL database.

If you need to ask questions, ask before you bid.

We can provide ALL logins (SSH, FTP, ADMIN, cPanel, WHM) as we have a dedicated server.

Good luck!

Skills Required

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online