We have a huge dataset (around 10GB after unzipping). We want to host it on an MS SQL server.
- We need help to set it up to begin with. Our vendor uploads the data on his secure site. We need a robust process to check for the updates, download the file, unzip it and then update the SQL table. The data gets delivered multiple times a month. The process should be such that it has to constantly look for a complete file posted on the vendor's site and then start the rest of the processes.
- The data needs to be indexed for faster querying.
- Provide us with MS SQL Server management studio interface to this server.
- We want the standard techniques to be used in the whole process so that we can manage things on our own once setup.
- We need support to debug any issue we might face - but we don't expect any.