In Progress

Automated web data scraping, extraction, parsing

This job is about developing a simple script that executes the following sequence of activities, either automatically at regular time intervals or when activated by the user.


1) check time and connect accordingly to several websites (some password-protected, some public) - should be a list / configuration file that the user can update easily and safely

2) go to several identified pages on these sites - should be a URL list / configuration file that the user can change easily

3) extract specific information identifiable through a column or table header

4) store this information in Excel and / or Google Spreadsheet files and / or text files (final decision to be based on technical feasibility)

6) save the files based on a predefined naming convention that reflects time and date

Suggestions are welcome in terms of technology choice. However please note that preference will be given to approaches that make it easy to maintain / modify the code and that reduce development time / cost.

For your proposal to be considered, you must IMPERATIVELY provide the following information:

a) please detail your prior experience with web automation projects, especially content scraping and parsing. please indicate if you have an existing library of code that you can re-purpose, and how it reduces the cost of your proposal.

b) please detail your prior experience with Windows / Excel automation projects. Please note that this is not a deal breaker if the output is a text file.

c) please describe what technology you recommend for this project, and why.

d) please specify cost range, and estimated completion and delivery timeline for this project.

If you need any further details to finalize your proposal, feel free to contact me!

Thanks in advance for considering this job and looking forward to your proposal.

Job keywords: .NET, Windows, Excel, macros, Excel VBA, script, scripting, automation, web scraping, data scraping, crawling, data extraction, data parsing, PHP, Python, Scrapy

Skills: .NET, Perl, Python, Visual Basic, Web Scraping

See more: automated web scraping, what is a web script, what is a job proposal, web scraping with vba, web scraping python 3, web scraping free, web development technology, web development projects in php, web development cost, web developing projects, web d cost, vba spreadsheet scraping, spreadsheet on web, simple projects on web development, scripting job, script google spreadsheet, scraping websites with python, scraping web content, scraping free, python or php for web development, proposal for web development, perl technology, password vba project excel, password protected excel, note on web development

About the Employer:
( 102 reviews ) NEW YORK, United States

Project ID: #666342