Scrape major online retailer

Using python and the scrapy framework adapt our existing or create a new scraper which will do the following:

Crawl [url removed, login to view] and scrape information about the reviewers

Invoking the program should look like:

$ scrapy crawl amazon_reviewers --set FEED_URI=[url removed, login to view] --set FEED_FORMAT=csv

The fields we want to see are:

website - URL of the user's website (if any)

name - The user's name on amazon

pic_url - Url to their profile pic (if any)

userid - Their amazon profile id

email - E-mail address

Rank - Reviewer rank on amazon

Location - Geographic location if listed

num_reviews - Number of total reviews

The output should look like this in csv form:


http://...url...,K. Harris,[url removed, login to view],,,10,, 5000

Delivery should include:

- Full source code

- Scrape of reviewers from the top level categories: Electronics & Computers, Toys Kids & Baby, Sports & Outdoors, Home Garden & Tools, Automotive & Industrial

Major challenge is finding these reviewers. Only 10,000 are exposed easily, the rest must be found by crawling through product pages.

The other major challenge will be actually doing the crawling. I have access to significant EC2 resources so I can help you once the code is ready, but I'd prefer it if you were able to crawl yourself and then just provide a data sample to verify.

This MUST be done using python and the scrapy framework as we will need it to work with the rest of our tech stack.

Will likely expand scope upon timely delivery of first project. Please include a link to your prior work, preferably a public github with python projects visible.

Price is negotiable.

Skills: Data Mining, Python

See more: project scrape, work from home with computers, work from home with baby, work from home with amazon, work from home sports, work from home on computers, work from home computers, top online email, tech challenge 2013, sample stack program in c, sample product profile, python source code github, python home work, python help online, project reviewer, program website in python, online work done from home, online tech, online reviewer, online python projects, online projects work from home, online projects to work from home, online projects on python, online code major, i want to work from home online

About the Employer:
( 0 reviews ) United States

Project ID: #4776444

12 freelancers are bidding on average $1061 for this job


I am an expert in scraping and can deliver the script as required

$1263 USD in 12 days
(15 Reviews)

Hello. Please refer to your inbox. Thanks

$1184 USD in 3 days
(10 Reviews)

I have done scrapping using Python and Scrapy for my own project.

$1250 USD in 15 days
(4 Reviews)

Ready start right now. Can deliver beta version in 3 days. Have reat expirience in scrapping & use scrapy framework in my projects.

$1388 USD in 7 days
(6 Reviews)

Hey, We have already worked on projects of such kind wherein you require to scrap full site on a page navigation basis. Looking forward to an opportunity to work, Regards, Sunny

$888 USD in 10 days
(1 Review)

Hi, We have expert in-house team of Python Programmers. Please check private message.

$979 USD in 25 days
(0 Reviews)

hi, Hi, Greetings of the day and thanks for sharing these details and allowing me to bid on this assignment. we have already worked on a similar project before and can deliver as u have mentioned in your requi More

$947 USD in 21 days
(0 Reviews)

HI ! We've amazon scraper ready . Just process the payment & get the scraper. Check This For More Details : [url removed, login to view]

$833 USD in 1 day
(0 Reviews)

I used to create a hotfile spider by scrapy,and I'm very interested in this project. You can content me for more details.

$1260 USD in 20 days
(0 Reviews)

Please see your pm.

$773 USD in 3 days
(0 Reviews)

Please check my Private message. We have something ready that it's exactly what you need.

$1111 USD in 1 day
(0 Reviews)

Hello. As a professional web scraper, I will of course build this in Python on the scrapy framework, and you can run it with that exact command if you'd like. Delivery will be in no more than 3 days, and I could do the More

$944 USD in 3 days
(0 Reviews)

I can do this and can provide you working demo if required.

$944 USD in 3 days
(0 Reviews)

I was actually working on a project like this, and all I have to do to the source code is to change a few variables, and choose different html tags. I'll pm you the source code, I can do it for $450 instead of your $7 More

$750 USD in 1 day
(0 Reviews)