Web Crawler with Variable Proxy

This project received 20 bids from talented freelancers with an average bid price of $136 USD.

Get free quotes for a project like this
Project Budget
$30 - $250 USD
Total Bids
Project Description

I am looking for a programmer to help develop a web crawler. The app needs to crawl a few websites on a weekly basis. Some of the websites have volume restrictions and require the ability to connect to a proxy server after so many requests. The output needs to be in the format of a .CSV file with headers. One of the websites requires a username and password before proceeding. Please see a couple samples below:

[url removed, login to view]

[url removed, login to view]

I would prefer that the application be written as a macro in VBA for excel but will except all other full applications with raw files. The applications needs to be equipped with error handling incase of a 404, 302, timeout, ect. I would also like to have a status bar or estimated time notification to allow the user to see that the application is in progress.

If you have read the full project details please include the word BLUE in your bid. I have had similar projects completed for less than $100 USD.

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online