Automated web data scraping, extraction, parsing

This project was awarded to kovmisa for $90 USD.

Get free quotes for a project like this
Project Budget
$30 - $250 USD
Total Bids
Project Description

This job is about developing a simple script that executes the following sequence of activities, either automatically at regular time intervals or when activated by the user.


1) check time and connect accordingly to several websites (some password-protected, some public) - should be a list / configuration file that the user can update easily and safely

2) go to several identified pages on these sites - should be a URL list / configuration file that the user can change easily

3) extract specific information identifiable through a column or table header

4) store this information in Excel and / or Google Spreadsheet files and / or text files (final decision to be based on technical feasibility)

6) save the files based on a predefined naming convention that reflects time and date

Suggestions are welcome in terms of technology choice. However please note that preference will be given to approaches that make it easy to maintain / modify the code and that reduce development time / cost.

For your proposal to be considered, you must IMPERATIVELY provide the following information:

a) please detail your prior experience with web automation projects, especially content scraping and parsing. please indicate if you have an existing library of code that you can re-purpose, and how it reduces the cost of your proposal.

b) please detail your prior experience with Windows / Excel automation projects. Please note that this is not a deal breaker if the output is a text file.

c) please describe what technology you recommend for this project, and why.

d) please specify cost range, and estimated completion and delivery timeline for this project.

If you need any further details to finalize your proposal, feel free to contact me!

Thanks in advance for considering this job and looking forward to your proposal.

Job keywords: .NET, Windows, Excel, macros, Excel VBA, script, scripting, automation, web scraping, data scraping, crawling, data extraction, data parsing, PHP, Python, Scrapy

Awarded to:

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online