In Progress

Web Crawl from Internet Archive

I'd like to gather some data for an academic project to study the electronic book market.

The Internet Archive (Wayback Machine) had crawled websites that are of interest to me in the relevant period, and I'd like your help to

(1) Crawl Internet Archive to save html pages of interest

(2) Extract relevant fields in the html to form a comma separated file ready for data analysis packages.

Task1: Crawl

The webpage of interest are product page of books or e-book reader devices in the following period, venue, and category:

Time period:

2010.1 - 2010.5 (one capture a day if available)


Amazon, Barns & Noble


Physical Book, Kindle/Nook book. (not textbook, newspaper, etc. )

Device itself: Kindle and Nook.

Books listed as bestseller, award winner, editor's picks, best books, book club, etc.

We can discuss whether it's easier to get all books or just the popular books.

Task2: Extract

Fields of interest: Title, author, publisher, # reviews, ratings, list price, discount price, price of other formats, whether listed as bestseller, sales rank, ISBN, category.


(1) Small sample - prefer to have a small sample by May 14th.

Amazon only, one day in mid March, one day in mid April, one day in mid May in 2010.

(2) Negotiable, but preferably completed before June 5th.

(3) Possible future projects to extract 2005-2013 if initial run goes well.

Skills: Web Scraping

See more: web scraping price, the best web page editor, textbook editor, scraping the internet, popular 20 discount, list of books by author, kindle sites, kindle book packages, kindle book club, internet market all, get reviews for amazon, best web scraping, best sites to get reviews, amazon price scraping, web scraping from websites, web scraping amazon, the period club, scraping amazon, period club, kindle reviews, internet projects, electronic projects, crawl data, crawl a we, club web

About the Employer:
( 1 review ) Boston, United States

Project ID: #4506529

Awarded to:


I have created many crawler like this one. Please check inbox. I am ready to work now.

$155 USD in 3 days
(3 Reviews)

6 freelancers are bidding on average $208 for this job


Hello, i have expertise in web scraping. If you are interested in my bid, please contact me. Best Regards.

$315 USD in 5 days
(51 Reviews)

pl check your inbox

$200 USD in 5 days
(25 Reviews)

Hi alicealisa, I can get the info you need. I have a lot of experience getting info from websites and I'm available immediately. For more info, please see my pm.

$100 USD in 3 days
(6 Reviews)

Scraping Experts Here. Check the message and contact us. Scraping samples are also attached.

$250 USD in 5 days
(6 Reviews)

Ready to go now, please check pm.

$220 USD in 3 days
(0 Reviews)

I know crawling with php and python . you can take me as crawler expert. check pm

$257 USD in 10 days
(0 Reviews)