Automated web data scraping, extraction, parsing
This project was awarded to kovmisa for $90 USD.Get free quotes for a project like this
Project Budget$30 - $250 USD
This job is about developing a simple script that executes the following sequence of activities, either automatically at regular time intervals or when activated by the user.
1) check time and connect accordingly to several websites (some password-protected, some public) - should be a list / configuration file that the user can update easily and safely
2) go to several identified pages on these sites - should be a URL list / configuration file that the user can change easily
3) extract specific information identifiable through a column or table header
4) store this information in Excel and / or Google Spreadsheet files and / or text files (final decision to be based on technical feasibility)
6) save the files based on a predefined naming convention that reflects time and date
Suggestions are welcome in terms of technology choice. However please note that preference will be given to approaches that make it easy to maintain / modify the code and that reduce development time / cost.
For your proposal to be considered, you must IMPERATIVELY provide the following information:
a) please detail your prior experience with web automation projects, especially content scraping and parsing. please indicate if you have an existing library of code that you can re-purpose, and how it reduces the cost of your proposal.
b) please detail your prior experience with Windows / Excel automation projects. Please note that this is not a deal breaker if the output is a text file.
c) please describe what technology you recommend for this project, and why.
d) please specify cost range, and estimated completion and delivery timeline for this project.
If you need any further details to finalize your proposal, feel free to contact me!
Thanks in advance for considering this job and looking forward to your proposal.
Job keywords: .NET, Windows, Excel, macros, Excel VBA, script, scripting, automation, web scraping, data scraping, crawling, data extraction, data parsing, PHP, Python, Scrapy
Looking to make some money?
- Set your budget and the timeframe
- Outline your proposal
- Get paid for your work
Hire Freelancers who also bid on this project
Looking for work?
Work on projects like this and make money from home!Sign Up Now
- The New York Times
- Wall Street Journal
- Times Online