Closed

Crawling/Scraping Engineer

We're looking for a scraping/crawling engineer to join full time our 3-people Crawling team for a 3-month period.

This team is responsible for extracting information from thousands of websites every day. Due to constant changes in some of these websites, a number of our extractors break every day, so we need to continuously adapt them. We also need to continuously create new extractors for new sources.

Our tech stack: scrapy + Selenium. We use JIRA for the management of tasks.

Requirements:

- fluency in English, both written and spoken

- scraping/crawling experience (3+ years) using scrapy + Selenium

- python programming experience

- experience with online services, such as AWS and Cloudera

Please apply only in case you're an individual (i.e. no agencies) interested in joining the team full time for the next 3 months. Links to GitHub projects that you've developed in the past and any other references will be much appreciated.

A technical interview will be conducted before awarding the job.

Skills: CSS, Python, Scrapy, Web Scraping, xpath

See more: web scraping, job scraping tools, web crawler api, web crawler, web crawling techniques, scrapy, web crawler tutorial, web crawler python, google job online data entry, job online php developer prado, getting job online live chat agent, engineer job sweden spanish, apply for a job online, system engineer job online, apply for job online graphic design, apply for job online in dubai for graphic designer, apply job online for graphic designer, apply typing job online, engineer freelance job online, find job online for web scraping

About the Employer:
( 0 reviews ) Munich, Germany

Project ID: #19991162

33 freelancers are bidding on average €17/hour for this job

chirgeo

Hi. We are working on a new web crawling platform which we think have some advantages over scrapy and especially Selenium, which is limited in many ways. If you are interested to find out more let me know and we can t More

€17 EUR / hour
(113 Reviews)
7.3
zekovicm

Hi there,I am Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and finish it within the More

€20 EUR / hour
(97 Reviews)
7.2
sky930320

Dear Sir. Glad to meet you. I'm Web developer specializing in web scraping crawling and indexing web pages. Skills: python, scrapy, selenium, requests, beautifulsoup, mechanize, lxml, urllib2, automation, bots, spide More

€15 EUR / hour
(28 Reviews)
6.4
schoudhary1553

Hello, I have gone through your job posting and become very much interested to work with you. I am an expert in this field. I have already completed several projects like this. For evidence you can see my profile. Pl More

€18 EUR / hour
(38 Reviews)
6.2
devops724

Hi There I have 10 years experience in Python and web development. 8 years developing Django sites and services. I did lots of projects in scrapy and crawler. i offer/accept lower price to get more rank. technology tha More

€23 EUR / hour
(70 Reviews)
6.4
Mickelson

Hi Nice to meet you. I have enough experience in python script. Below the libraries are I used in past project. selenium, pandas, matplotlib, lxml, beautifulsoup, scipy, and other useful libraries. I have written More

€16 EUR / hour
(51 Reviews)
5.9
stead121

Hello, Glad to meet you I have strong experience and skill in crawling and scraping as you could see in my recent reviews. Scrapy and selenium is my major skill for crawling. I am also so familiar with team work thro More

€12 EUR / hour
(51 Reviews)
6.0
yesikov1224

[login to view URL] Credit is my motto. I am expert web scraping. I can avoid recaptcha. I can do your job with BS4 ,Perl script, Seleinum framework of python. I can do any project in your demand completely by my good experiences More

€15 EUR / hour
(51 Reviews)
6.0
yongjin818

Dear As an expert in web scraping, I developed 1000+ spiders using python, scrapy, php,nodejs,c# and proxies. And I scraped data and download images, files for CEOs and output result as csv, json, xml(yellow, ebay, w More

€15 EUR / hour
(68 Reviews)
6.0
devexpertsteam

Hello, We are IT-Engineers and Business Analysts, creating complete web and mobile solutions, for already more than 15 years, supporting our products with custom digital marketing strategies and instruments. Scra More

€70 EUR / hour
(1 Review)
5.6
susanna2018

Hi, Client!! @$@$@$PLEASE CONTACT ME. I CAN DO IT WONDERFULLY.$@$@$@ ##% Your idea is attracted me to your world.%## I am Python expert. i have various skills. @@PLEASE GIVE ME CHANCE TO PROVE MY SKILLS@@ :-): More

€16 EUR / hour
(34 Reviews)
5.6
abhilashtv

Hi, ➲ 10+ years of full-time experience in Python / Django with 50,000+ Upwork hours billed and 50+ successful Python projects ➲ Upwork Top 10 Certification for Python and Django ➲ Guaranteed Results Policy: Pay onl More

€12 EUR / hour
(13 Reviews)
5.5
smsaurabhv

‌Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHO More

€16 EUR / hour
(60 Reviews)
5.2
Thesynapses

Note: We are an Agency but still placing the bids because in data fetching we have developed a product "Dragon SRCH, which may help you run the daily routine job. Dragon SRCH Platform is a cloud-based research eng More

€20 EUR / hour
(3 Reviews)
5.1
albertpopov46

Hi, Dear. I have many development experiences, getting work about. As I am interested in your project, I want to do work it. I've read the project description and definitely can develop your task. If you hire me, I w More

€12 EUR / hour
(18 Reviews)
4.7
shenji108

Hello, I'm Android (since 2012) and web (since 2013)developer. I can speak in English freely. I tend to keep myself up to date with important changes in the Android world and web frameworks and regularly look into ne More

€16 EUR / hour
(7 Reviews)
4.7
artistixeit

""I can do it"" Food/Restaurant/order/Delivery/Eat: [login to view URL] [login to view URL] [login to view URL] [login to view URL] 3D Cart: https: More

€14 EUR / hour
(2 Reviews)
4.7
KongHyongRan

Hello, Webcrawls is may main skill !!! As I have accumulated gained rich experience with web applications, I'm pretty sure I can deliver satisfactory product for you. I can give you better suggestion based on detail More

€16 EUR / hour
(13 Reviews)
4.4
abdullahbaig052

Hello Sir, Talent is not limited to Europe sir. 100% fluent in English. A sample can be provided once you inbox. I am scrapy specialist. 100% fluent in English. I have been working in scrapinghub for quite a while. Wo More

€18 EUR / hour
(5 Reviews)
3.9
oswaldoodavidal

Hello, I have experience in web scraping with Python. I can use Selenium, Scrapy, BeautifulSoup and Requests to make the best web scrapers! I hope to work with you!

€16 EUR / hour
(2 Reviews)
4.1