We are looking for an expert web scraper to complete a number of more scraping projects. The websites being scraped are sophisticated and such the scraping infrastructure and strategy must be sophisticated to avoid having our scrapers blocked. Only accept this job if you have extensive experience setting the up the server infrastructure.
1. Analyze website from which we are scraping. Determine best way to gather data. Ideally, scrape their JSON if available.
2. Build scrapers using Python and provide the data in JSON format. We will provide the data and JSON format to create.
3. Build a server infrastructure on Digital Ocean using a round robin or equally sophisticated setup.
4. Utilize a squid proxy or equally sophisticated setup.
5. All code will be owned and transferred so that we can completely manage moving forward. The entire process should have no external dependencies outside our control.
Sites and data we will be scraping will be provided upon hiring. In order to be considered for this position, please send an example of a project with the infrastructure you setup and your github and/or project code where we can see your scraping code.
22 freelancers are bidding on average $229 for this job
Hello, I have experience in web scraping with Python. I can use Selenium, Scrapy, BeautifulSoup and Requests to make the best web scrapers! I hope to work with you!