Completed

File Parser to Scan [login to view URL] Files and Calculate Unique IP Addresses

Problem:

I have a large number of [login to view URL] files that each contain csv data. There are about 250 of these files and each one is about 100mb in size (compressed). The CSVs contain lists of IP addresses in column A and I would like to know how many unique IP addresses there are in the files (total, not per file).

Solution:

I would like a file parser written in python that will scan the [login to view URL] files and tell me how many unique IP addresses exist within the csv data. By this I mean total unique IP addresses amongst ALL the data files, not just the unique IP addresses within each individual file.

Thank you for your assistance. Please include a brief description in your PM along with your bid so I can tell that you actually read the project description instead of using an auto-bidder to bid on the project. Just a sentence or two will do.

Skills: C Programming, Database Programming, Java, Python, Software Architecture

See more: python read csv example, open csv file terminal mac, python read csv line by line, how to open csv file in command prompt, how to create a csv file in python, python read csv column into list, best way to read large csv file in java, how to extract data from csv file in python, link file parser, java file parser find text files, vb6 text file parser project example, parse mt940 file parser, data file parser, text file parser, php txt file parser, edi file parser, xml file parser database, vb6 csv file parser, text file parser csv output, as3 csv file parser

About the Employer:
( 92 reviews ) Fort Worth, United States

Project ID: #18989668

Awarded to:

osmanaydin

Hello sir, I can parse your files for IP adresses as you like. Please take a look at my reviews and portfolio. I am interested in your project as well. I would like to discuss details via pm. I look forward to heari More

$100 USD in 1 day
(23 Reviews)
4.8

18 freelancers are bidding on average $71 for this job

kaloyan13

A Bash script will be a better/faster option. Will extract each file, append the IPs to a temporary files, then we will unique sort them, using sort -u -k...

$40 USD in 1 day
(321 Reviews)
6.9
Mickelson

Hi Nice to meet you. I checked your description. I have some similiar script for processing .gz files. I will use numpy library for your project. Just get total unique count is not problem. My question is how you More

$100 USD in 10 days
(66 Reviews)
6.3
AhmadSameer

Hey there, I can develop the CSVs parser to count unique IP addresses. I'm a System Engineer with coding skills. I had developed tons of Python scripts. Would you share more details? Regards.

$100 USD in 10 days
(19 Reviews)
4.6
ukgolang

Hi, I'm not strong on Python, but could write this for you in Golang and compile it for Windows or Linux. I've done work parsing and analysing BGP router views logs, so have I previous experience working with large amo More

$55 USD in 1 day
(6 Reviews)
4.3
alexejshevchenko

Hello. I can create an app for Windows using C#. You will upload all gz files and it will calculate unique IPs inside. But you should have RAM size more than summ of all csv files

$100 USD in 5 days
(7 Reviews)
3.3
pushpreethanspal

Hey there. Your project looks straightforward. I have done a lot of work with Python and csv files before, and even though I haven't worked with gzipped data, I don't see that being a problem. Even though 100Mb isn't m More

$100 USD in 7 days
(3 Reviews)
3.4
namitkhandelwal

Its about your project you posted: things can also be done in Java. We can do all the task you mentioned in here. hello, warm greetings! I am a Java developer working on Java technology since 7+ years havin More

$70 USD in 10 days
(10 Reviews)
3.3
KongHyanJa

Dear Client!!. I'm an expert of programming Thank you for your attention I am a Programmer with 7 years of experience in full circle of software development. I am interesting in Python and Statistics,Web Scrap More

$61 USD in 10 days
(9 Reviews)
3.3
VnNorthStar

Hi I checked your requirement, I'm sure I can do it well. Gzip library in Python supplies some methods to manipulate with a gzip big file, read all lines from gzip files. Please give me an opportunity. I will do More

$30 USD in 5 days
(29 Reviews)
3.8
edgarizotov

Greetings. There are two ways how we can solve this task: 1) Dump all the data to SQLite DB and fetch all unique IPs via SQL 2) Iterate over every CSV file and add to global variable only unique IPs, but it will be m More

$66 USD in 2 days
(2 Reviews)
1.0
teknosoft28

"Hi, Hope you are doing well! Thanks for sharing your project requirement with us. It will be our great pleasure to work on your project. I have checked your requirement, yes we can do it, because we already work on si More

$97 USD in 7 days
(0 Reviews)
0.0
anatole7

Hi. Your task interesting for me. Python is my third language and I look out more practice for it everywhere. I will glad to do it task for you. p.s. my bid is 55 and approximately 1-2 day for work, testing an More

$55 USD in 2 days
(1 Review)
0.0
ramagni

Hi , Hope you are doing well. Should be able to deliver this project in a day max . I understand the requirement . (ip address-is it ipv4 or ipv6 ). I will be using python as language for implementing this. We can More

$66 USD in 1 day
(1 Review)
0.0
jayantavkumar

I am a database / Business intelligence architect and having more than 14 years of experience in IT industry. I can achieve it using SSIS ETL tool. Let me know if you are ok then we can talk further.

$88 USD in 2 days
(0 Reviews)
0.0
Nitin216

Have been working in Python for past one year.. Also contributed to an open source org Symy in past Relevant Skills and Experience Previously worked on Python built Sympy(Open Source) Used Python for ML purpose too.

$61 USD in 10 days
(0 Reviews)
0.0
skyhaubrich

How's it going? No, this is not an auto-bid. I hope this is enough to prove I've read your project description. I'm Sky, a German Development Enginner with more than 6 years experience in Software Engineering. I More

$35 USD in 0 days
(0 Reviews)
0.0
ahmedelhadidy95

I am an Embedded software Engineer working at Stakrbits company, programming with C , C++ and python

$55 USD in 10 days
(0 Reviews)
0.0