Scrape major online retailer
This project received 14 bids from talented freelancers with an average bid price of $1037 USD.Get free quotes for a project like this
Project Budget$750 - $1500 USD
Using python and the scrapy framework adapt our existing or create a new scraper which will do the following:
Crawl [url removed, login to view] and scrape information about the reviewers
Invoking the program should look like:
$ scrapy crawl amazon_reviewers --set FEED_URI=[url removed, login to view] --set FEED_FORMAT=csv
The fields we want to see are:
website - URL of the user's website (if any)
name - The user's name on amazon
pic_url - Url to their profile pic (if any)
userid - Their amazon profile id
email - E-mail address
Rank - Reviewer rank on amazon
Location - Geographic location if listed
num_reviews - Number of total reviews
The output should look like this in csv form:
http://...url...,K. Harris,[url removed, login to view],,,10,, 5000
Delivery should include:
- Full source code
- Scrape of reviewers from the top level categories: Electronics & Computers, Toys Kids & Baby, Sports & Outdoors, Home Garden & Tools, Automotive & Industrial
Major challenge is finding these reviewers. Only 10,000 are exposed easily, the rest must be found by crawling through product pages.
The other major challenge will be actually doing the crawling. I have access to significant EC2 resources so I can help you once the code is ready, but I'd prefer it if you were able to crawl yourself and then just provide a data sample to verify.
This MUST be done using python and the scrapy framework as we will need it to work with the rest of our tech stack.
Will likely expand scope upon timely delivery of first project. Please include a link to your prior work, preferably a public github with python projects visible.
Price is negotiable.
Browse Related Skills
Other things people do on Freelancer
Looking to make some money?
- Set your budget and the timeframe
- Outline your proposal
- Get paid for your work
Hire Freelancers who also bid on this project
Looking for work?
Work on projects like this and make money from home!Sign Up Now
- The New York Times
- Wall Street Journal
- Times Online