I have a large number of [login to view URL] files that each contain csv data. There are about 250 of these files and each one is about 100mb in size (compressed). The CSVs contain lists of IP addresses in column A and I would like to know how many unique IP addresses there are in the files (total, not per file).
I would like a file parser written in python that will scan the [login to view URL] files and tell me how many unique IP addresses exist within the csv data. By this I mean total unique IP addresses amongst ALL the data files, not just the unique IP addresses within each individual file.
Thank you for your assistance. Please include a brief description in your PM along with your bid so I can tell that you actually read the project description instead of using an auto-bidder to bid on the project. Just a sentence or two will do.
18 freelancers are bidding on average $71 for this job
A Bash script will be a better/faster option. Will extract each file, append the IPs to a temporary files, then we will unique sort them, using sort -u -k...
Hey there, I can develop the CSVs parser to count unique IP addresses. I'm a System Engineer with coding skills. I had developed tons of Python scripts. Would you share more details? Regards.
Hello. I can create an app for Windows using C#. You will upload all gz files and it will calculate unique IPs inside. But you should have RAM size more than summ of all csv files
I am a database / Business intelligence architect and having more than 14 years of experience in IT industry. I can achieve it using SSIS ETL tool. Let me know if you are ok then we can talk further.
Have been working in Python for past one year.. Also contributed to an open source org Symy in past Relevant Skills and Experience Previously worked on Python built Sympy(Open Source) Used Python for ML purpose too.