Upload large zip or csv files into mysql on local computer for further treatment in c#
This project was awarded to uconomixtec for $240 USD.Get free quotes for a project like this
Project Budget$30 - $250 USD
I want a program in C# / MySQL, that would be able to:
upload about 50 zip or CSV files. These files contain historical financial data and are very large for a total size of about [url removed, login to view] zipped. I can send you a file example if you want.
1/ import files from a local computer. The files are either in ZIP or csv (a file example can be provided if necessary).
Table format (a table example can be provided if necessary).
orig_id bigint (20,0) not null primary key
id_minute bigint (20,0) not null
desc varchar(20) not null
datetime varchar(25) null
v1 decimal(10,5) null
v2 decimal(10,5) null
2/ sort data by datetime ascending
3/ remove any duplicate. duplicate if datetime & v1 & v2 in above/below records are equal
4/ add orig_id to every record
5/ group by minute (beginning & end of minute).
a) Please also allow possibility to only group by end of minute (or beginning of minute) instead of having beginning & end of minute values
b) Please also allow possibility to group by a variable number of minutes
6/ ad id_minute to every record by minute
7/ merge new data with historical data
8/ since process is very long, provide ability to text/email when each process is finished
The reason you will need to process file by file instead of importing all files together and doing the process only once is because the file size would be large to freeze the computer.
Once the 50 files are uploaded into mysql, design process to upload a monthly file to upload and do the same steps to then append to the past data.
Browse Related Skills
Other things people do on Freelancer
Looking to make some money?
- Set your budget and the timeframe
- Outline your proposal
- Get paid for your work
Hire Freelancers who also bid on this project
Looking for work?
Work on projects like this and make money from home!Sign Up Now
- The New York Times
- Wall Street Journal
- Times Online