We are chasing a skilled coder to embark on the first stages of a new project. To begin with the desired individual must contain a high level of understanding into the following.
- Automate the retrieval of data from several websites - RSS Feed \ CSV \ Flat File
--- This will be either a daily or weekly process
--- This will also include update RSS feeds\files and deleted feeds\files
- Manipulate all files into a single format to be specified.
- Automate the adding of all files into a single table within SQL.
--- The injection of custom columns maybe required.
- Document the process of the above sites for record keeping and for future steps when adding future sites.
The download, manipulation and add/altering of sql is to be 100% automated with an email to be sent with the following:
- Total records added, total records removed, total errors.
The table will contain initially 1-2 million lines, with updates occurring daily and\or weekly. As such we are requiring an individual to implement a best practice for limiting the work load on the SQL platform as future stages of the project will become available for queries via external websites.
Prior to go live, you will have remote access to a Windows 2008 R2 VM Server with SQL 2012 to which the project work will be completed.
This project will contain several stages and we are looking for someone to continue working through all stages of this project.