Looking for a specialist in web site crawling
This project received 4 bids from talented freelancers with an average bid price of $16 USD / hour.Get free quotes for a project like this
Browse Related Skills
Other things people do on Freelancer
Project Budget$8 - $15 USD / hour
We are looking for freelancers who are specialized in web site crawling. We are working on several projects which require full crawling of web sites like e.g. http://www.parlament.ch. For large web sites we typically define several subsites which can serve as improved starting points for the crawler. The results should be the complete texts contained in the web site. Text in PDF files or in HTML-tables also need to be crawled and available in the result. Once the crawler is correctly set up for a specific site, we typically expect a periodic crawling of the site contents (e.g. once a week).
We are looking for someone who is experienced in this data gathering process and can manage all steps (setup crawler, improve crawler, manage document content update, transfer data to our server).
Crawling can be done with Apache Nutch or other crawling softwares which the specialist recommends.
Looking to make some money?
- Set your budget and the timeframe
- Outline your proposal
- Get paid for your work
Hire Freelancers who also bid on this project
Looking for work?
Work on projects like this and make money from home!Sign Up Now
- The New York Times
- Wall Street Journal
- Times Online