Enhance Bash Script: CSV parser


I have a bash script that accepts an input csv file, allows for several command line options such as delimiter and rolls up details on the level of duplication of fields on each column of data into a report. It can handle files with millions of rows by pulling one column of data into memory at a time, writing a temporary file of the most duplicated fields (e.g it does something similar to this for each field in the input file: cut -d\, -f1 [url removed, login to view] | sort | uniq -ci | sort -nr | head -20)

The script requires some modifications and enhancements including:

* Better parsing of csv files. It handles files with rows like: foo,bar,baz or "foo","bar","baz". But it has issues parsing "foo",123,"bar".

* Certain fields require special parsing. e.g. I would like the option to treat [url removed, login to view] and [url removed, login to view] and all it's variants as the same URL and therefore get counted as a dupe.

There are a few other tweaks of similar nature that I would like to get incorporated into the script which we can discuss. Looking forward to hearing from a bash guru.


Skills: Shell Script, UNIX

See more: bash csv parsing column, csv bash, bash script csv, enhance script, bash script csv parsing, bash csv parser, writing guru, writing a script, writing a field report, writing a bash script, url guru, parsing input, options guru, option guru, guru writing, guru unix, guru report, guru on line, bash script csv parser, shell bash, bash file, bash 0, uniq, f1, bash script

Project ID: #4137021

Awarded to:


If you would accept the script in Perl as well (can't reaaly think of an environment where bash works but Perl doesn't - are you really operating in such a beast ...?), I'd be glad to help - and you'll get a robust, fl More

$59 USD in 1 day
(5 Reviews)

11 freelancers are bidding on average $72 for this job


Hi I am a CISSP,RHCE,CCNA,MCSA,Linux+ and a CEH. I do have 7+ years experience in System Administration . I have experience in a high availability environment with 100+ servers and more than 500 000+ subscribers, I kno More

$80 USD in 3 days
(2 Reviews)

Hi, Please see PM.

$50 USD in 1 day
(2 Reviews)

Hi There! My bid on your project is not the shortest timespan nor is it the cheapest by far, you were however looking for a bash "guru" and I believe I fit that bill. I script in bash almost every day and I know it bet More

$250 USD in 5 days
(0 Reviews)

Hi, I can do this .

$60 USD in 2 days
(0 Reviews)

Hi! This easy task for me. I love shell code. Usually I write in posix standard (or e.g. use coreutils with posix shell). But I also can write most part in bash (exactly how much will be on the bash, and what with cor More

$30 USD in 2 days
(0 Reviews)

Hi I can do this for you. Have 6 plus years of experience in bash and very comfortable with regexp which you need for your script. Regards Jaspreet Sahota

$30 USD in 2 days
(0 Reviews)

I can do it.Just one day.

$30 USD in 1 day
(0 Reviews)

Hello, I can do this for you. Please go through my PM. Thanks kiran

$75 USD in 7 days
(0 Reviews)

would like to work on this projects details sent you on pvt message.

$31 USD in 2 days
(0 Reviews)

ready to start

$100 USD in 7 days
(0 Reviews)