Find Jobs
Hire Freelancers

Write a webCrawler to extract basic data from site (Long term Job possibilities)

€30-250 EUR

Closed
Posted about 7 years ago

€30-250 EUR

Paid on delivery
Hi, We are looking for a programmer that can write/configure a webcrawler to crawl a website and retrieve the records list. We are thinking to use Apache Nutch (with selenium) to do the crawling (other possible). These records need to be parsed, so the information (id, title, introtext, date,...) can be stored in a database. If this job is done with success, the crawler can be extended to crawl more websites (100+) in a next project. We are looking for a long term partnership with good software engineers. Our company consists of 20+ software engineers and is looking to expand the team via outsourcing.
Project ID: 13263896

About the project

14 proposals
Remote project
Active 7 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
14 freelancers are bidding on average €146 EUR for this job
User Avatar
Dear sir, I am scraping expert, I have did too many scraping projects, please check my reviews then you will know. Can you tell me more details? then I will provide example data/script for you. Thanks, Kimi
€164 EUR in 5 days
5.0 (256 reviews)
7.3
7.3
User Avatar
Hello, I want to show you all relevant Demo and Designs which is similar to your project completed previously. To make sure about the requirement set and customizations, I want to discuss this project with you further on personal chat. Let me know the best suitable time for you to schedule the meeting, Feel free to message me at any time, I use to be online 24x7 on Freelancer so probably you will get a quick response from my end. Following are my Expertise Area: 1)PHP with CodeIgniter and Laravel Framework. 2)Node JS 3)Angular JS 4)Mobile App Development Thanks
€257 EUR in 10 days
5.0 (27 reviews)
7.0
7.0
User Avatar
Hello Sir, We've done a number of web scraping projects for our clients. We have scraped many directory websites including yellowpages, yelp and e-commerce websites including amazon, walmart etc and many more. We can deliver the data very quickly. We use proxies with IP rotation to avoid being detected as bots. We use python with wget, scrapy, urllib and other tools to fetch webpages and parsers like HtmlXPathSelector, regular expressions etc to extract information from the html. We have the right skill set to do this job effectively and within time and would like to discuss more about this opportunity. Looking forward to hear from you. Thanks, Shiv Agrawal
€73 EUR in 1 day
4.9 (94 reviews)
6.5
6.5
User Avatar
Hi there, I am java developer. I have strong knowledge in web crawling, web scrapping and bots. I have already developed bots and scrappers for Facebook, Instagra, Twitter, Property sites and many more. You can view them on my profile. This must be an easy job for me. Message me, we can discuss over chat.
€250 EUR in 3 days
5.0 (42 reviews)
5.8
5.8
User Avatar
Hello! My name is Warren Schultz, and I am a professional programmer with many years of web scraping experience using Python. I have read your project description, and I can create the program you have described. To do so, I will use the libraries Requests (for web automation tools), BeautifulSoup (for parsing html), and Selenium (for working with AJAX and Javascript). I have written very similar programs in the past, and I would be happy to show examples. Please contact me so we may speak further and so I can send files. Thank you for your consideration.
€155 EUR in 3 days
4.9 (36 reviews)
5.9
5.9
User Avatar
Hello, My name is Rui Pimentel and I'm an expert in development of web crawling and web automation tools with more than 10years of experience in the field. I've read your entire project description, understand what you need, and can do it in your budget. The tool would be developed from source in PHP using only the cURL library, I can assure you'll be very pleased with my solution. I'll be waiting to hear from you and I hope we can build a great relationship. If you check my last feedbacks you will understand I'm a very talented freelancer and communication won't be an issue, I'll not make you lose your time! Thanks, Rui Pimentel
€105 EUR in 3 days
4.8 (35 reviews)
5.9
5.9
User Avatar
Hello Sir, I have good experience in scrapping data from specified website. I can also give demo of the same. Please send me message for further information.
€144 EUR in 2 days
5.0 (16 reviews)
5.4
5.4
User Avatar
Hi I'm a java developer and I'm interested in this project. I have some experience in web scrapping/automation, including (selenium). Contact me to discuss more details.
€100 EUR in 5 days
5.0 (1 review)
1.8
1.8
User Avatar
High skill in data mining and scraping. More finished projects: any social networks include g+, vk, facebook, instagramm, twiitter e.g. and other sites. I keep data in a convenient format. Have experience in work with python, selenium.
€55 EUR in 2 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Dear Sir, Please find my expertise below, WEBSITE AUTOMATION - Selenium WebDriver, HTMLUnit, Sikuli, Phantomjs, Protractor, Java, C#, Python, Cucumber, JSoup, JUnit, AWS, Maven, TestNG, Jenkins 1. Designed and implemented different automation frameworks like Page Objects, Keyword Driven and Data-Driven and Hybrid using Selenium WebDriver and Java. 2. Created a BDD framework using Cucumber and configured a parameterized build in Jenkins for Continuous Integration. 3. Executed the Automation scripts as scheduled builds from Jenkins in SauceLabs, BrowserStack and Selenium Grid. 4. Created automation framework using Selenium WebDriver, TestNG, and Java for different scenarios, identify unique properties of objects, created user defined functions, reusable actions, and managed shared object repositories. 5. Automated the standalone application using Sikuli and helped in saving 20% of the manual testing and execution time.
€155 EUR in 3 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of BELGIUM
Brugge, Belgium
4.9
11
Payment method verified
Member since Mar 1, 2017

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.