Create a scraper / script / crawler to extract product data from an online shop - go through all products - export in csv or excel

In Progress

Description

Dear freelancers,

we need an effecient web scraper, which we can run on one of our own servers. WE ARE LOOKING FOR AN EXPERIENCED DEVELOPER - work must be flawless!

Following should be done:

The website to scrape/crawl is: [url removed, login to view]

--> It is an online shop with almost 80k products. The scraper should do the following: It should start with these main top level categories:

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

And should scrape EVERY SINGLE product within these categories (as said, something around 80,000 items).

The following information should be exported from EACH product - please use this url to understand different items explained below: [url removed, login to view]:

1) URL (e.g. "[url removed, login to view]")

2) Breadcrumbs (e.g. "Início > Masculino > Esporte Masculino > Calçados > Tenis")

3) Brand Name - located above product name (e.g. "Puma")

4) Product Name (e.g. "Tênis Puma Axis 2 Branco")

5) Image URLs --> ALL Images in product page --> USE default resolution (not zoom image) of ~275px × 400px (e.g. "[url removed, login to view] ; [url removed, login to view] ........ etc etc")

6) Current Price (e.g. "99,90")

7) Old Price - if applicable (e.g. "199,90")

8) Payable rates - if applicable (e.g. "5 x 19,98")

9) Available sizes: (e.g. "38, 39, 40, 41, 42, 43")

10) ALL Available Data in the tab "Detalhes do produto" --> Data here is:

--> A) a short text description AND

--> B) a list with multiple different entries (NOTE: products do not always have all these entries --> compare [url removed, login to view] versus [url removed, login to view]):

--> List items could be:

- Description (plain text above actual list)

- SKU (e.g. "RA870APM16PQL

- Modelo (e.g. "POLO RALPH LAUREN 89460PRL")

- Material (e.g. "Algodão")

- Composição (e.g. "100% Algodão")

- Cor (e.g. "Preto")

- Lavagem (e.g. "Lavar a mão")

- Medidas (e.g. "Ombro: 17cm/ Manga: 23cm/ Tórax: 116cm/ Comprimento: 76cm")

- Categoria (e.g. "Premium Masculino > Roupas > Pólos > Pólo Manga Curta")

--> That is all data we need for EACH product

***NOTE*** --> We will need to run the script MULTIPLE times per week: SO: The script MUST be effecient an FAST. The data should be extracted and then saved on the server (in csv or any other excel importable format). The script should possible to be run on OUR server.

***NOTE*** --> We are looking for a long term developer - we will not just need ONE script, BUT we will need similar scripts for 10 different online shops. SO: We are looking for somebody to then also develop other scripts.

Please get in touch if you have any questions.

Thank you very much,

Dan

Skills: PHP, Software Architecture, Web Scraping

See more: online crawler script, create web scraper extract data web page, www freelancers com br, work online as software developer, which software we need to develop a website in php, website for freelancers web developer, web developer rates, web crawler developer, web crawler architecture, top shop, top 10 freelancers website, top 10 excel freelancers, software developer shops, software developer rates, software developer freelancers, rates for software developer, polo. ralph lauren, php to jpg online, php developer work online, online web page developer, online web crawler, online manga, online architecture work, one page manga, modelo html 5

Project ID: #5073992

Awarded to:

name63

Hi! I can write a scrapper in perl, python or php. Perl is the fastest option. Will you run this script on Linux?

$99 USD in 3 days
(3 Reviews)
2.3

22 freelancers are bidding on average $203 for this job

mantislin

Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi

$157 USD in 4 days
(179 Reviews)
6.9
cheapexcell

Hello, I am an expert in web scraping. I have done many similar jobs. Ready to start. Thanks, Alex

$515 USD in 10 days
(168 Reviews)
6.7
acesolution

Hello Sir , I am highly interested in this project please provide me a chance to work for you....please message me i will give you demos of my scrapping work .........Thanks

$263 USD in 4 days
(114 Reviews)
6.6
ehsankayani

HI I am an expert in making scripts and most of my script use threading so they are extremely fast. Are you ok with a c# script that will run on windows?

$147 USD in 3 days
(56 Reviews)
6.4
ecartsolutions

Hello Sir The requirements are very clear in your project description. I am a PHP-Mysql expert and can help you with the data scrapping work. I can provide you with a sample of the scrap data within the 12 hrs. Plea More

$250 USD in 3 days
(251 Reviews)
6.3
gangabass

Hi, I'm expert in web scraping that's why I'm sure you'll be impressed with my work. I'm talking about the program which you can run every day and it will collect all info you need from the target site. The program More

$155 USD in 3 days
(164 Reviews)
6.1
uumairkhalid

Hi.. Expert web scraper/Data Minor here. Interested in your project. I assure you 100% accurate and good quality work. I have too too scraping experience. let me know if you want sample before awarding. lets start. Reg More

$315 USD in 3 days
(57 Reviews)
5.9
aoefmpes

Hello Sir, I have experience on web scraping, i can write script to do it quickly, pl ping me ,, i will send you sample data thanks

$244 USD in 7 days
(50 Reviews)
5.3
jitendraparmar07

Hello sir, I'm very pleased to submit this proposal to you. I'm software developer. I can easily write bot / scraper for your purpose. I'll write bot/scraper that scrapes data from website and import CSV /E More

$185 USD in 4 days
(39 Reviews)
5.3
uumarkhalid31

hi, i scraped ehops ad ecommerece websites many time and i am also interested in this project, let me do this work with perfection and accuracy thanks

$147 USD in 3 days
(29 Reviews)
5.1
tanveerjavaid

I have written some big scrappers while doing job in USA base software house to scrape craiglist so I can work on this project.. thanks

$150 USD in 0 days
(21 Reviews)
5.1
freelance4hire80

Hi Dan I'm expert in web scraping, and you can refer to my freelancer profile for similar jobs. My solution: 1. The crawler script written in Perl 2. The crawler write data into MySQL database. you can export out More

$222 USD in 3 days
(16 Reviews)
4.8
M5L2764K

Hi, I have a lot of experience building scrapers using Python, Scrapy framework and running them on Linux. Among other things Scrapy runs in parallel (gets results faster) and can use proxies/Tor (to avoid IP blocks More

$200 USD in 3 days
(5 Reviews)
4.5
RajakScripts

Hi, It can be harvested using regex & php curl lib, if I have a chance, I may give you a demo (if this project still open within 15 hours from now on because it's 18:30 in here already, so the demo may not finished More

$211 USD in 7 days
(10 Reviews)
4.0
anewstarting

EDIT : - I do have many more feedbacks than what you see here..:) Hi... Pleas ebe asured that I am experienced Java programmer...I have been working as a freelancer for more than 5 years and currently I am working More

$277 USD in 7 days
(7 Reviews)
3.1
computerpsycho

Hello Sir/Madam I have 3+ years of experience in web scraping. I have scraped more than 600 websites. I can do this for you with utmost accuracy and in time. I have the No.1 web scraping software. My bid price is More

$30 USD in 3 days
(6 Reviews)
2.9
markomarko0307

Hi there, i am using python scrapy module for scraping data from the site, for over 2 years i am scraping data for one danish company, some of the sites i scraped are: disney.com, express.com, spanx.com ,... and More

$222 USD in 3 days
(0 Reviews)
0.0
denisfd23

Propunerea nu a fost încă furnizată

$155 USD in 3 days
(0 Reviews)
0.0
bqureshi103

Hi, I will provide you a PHP script which will collect all the data you need. Let me know if you have any questions. Regards Bilal

$200 USD in 3 days
(0 Reviews)
0.0
urfanalimov

Hello, I have done similar scraping jobs for Russian guys who were scraping baby and junior clothing sites in turkey. They were ripping catalogs on Turkish sites and placing products on their own site in Russia. I h More

$166 USD in 3 days
(0 Reviews)
0.0