Completed

Matching System

1. There are a series of json file each around 15GB contains (IP+port)

2. There are series of json files each around a GB contains (traceroutes)

3. There are 3 group of csv files, each group contains 30 csv file

1st step

----------------

1. Load all the Json (1) in a MySQL table and delete the duplicates. (tbl_ports)

2. Load all the Json (2) in a MySQL table as they are traceroute IP's, Ignore the first two hop and last hop and store the rest in a SQL table. (tbl_tracceroutes)

3. Load each group of CSV file in a seprate table in MySQL and and add new column to this table and write the name of the csv file. (Do this step for the two rest of csv files group)

group1(tbl_xxx) - I will tell you the exact name when you start working

group2(tbl_yyy) - I will tell you the exact name when you start working

group3(tbl_zzz) - I will tell you the exact name when you start working

2nd Step

-----------------

1. in tbl_zzz, tbl_yyy, tbl_xxx there is a column contains subnetmask(IP+Cidr), take each of that (IP+Cidr) and look on tbl_ports on column IP, if any IP there matches to any (IP+Cidr) of tbl_zzz, you do this process for all of (IP+Cidr) for table tbl_zzz.

Create a new table (report_zzz), take some column from tbl_zzz (I will tell you which columns) and add a new column (port). for the ones that you find match write in report_zzz that row of that plus the port number you find for that (IP+Cidr).

** DO this process for tbl_xxx and tbl_zzz exactly the same way.

2. from the table report_zzz take the (IP+Cidr) data and look if any IP in tbl_traceroute match to that subnet, add new column to report_zzz named as (seen), if you find match write yes and if you don't find match write no.

** DO this process for report_xxx, report_yyy and report_zzz.

Requirements:

optimized code

As it is relatively big data, it should be fast enough

Bug free

Dynamic style

Skills: MySQL, SQL, Python, Network Administration

See more: joomla template matching phpbb style, php script matching system, auto matching system, order matching system, trade order matching system, php matching system, schedule matching system, nyminutedating matching system freelancer, web development trade matching system, online matching system login, create online matching system, profile matching system, job matching system php, design controller automatic paint color matching system, controller automatic paint color matching system circuit, digital logic controller automatic paint color matching system, automatic paint color matching system logic

About the Employer:
( 2 reviews ) Berlin, Afghanistan

Project ID: #20101564

Awarded to:

                                                    kozachenkodr
                                                's Profile Picture"                                                    >
                                                </a>
                                            </figure>
                                        </div>
                                        <div class=
kozachenkodr

Hi. I have very strong experience in dealing with big data (json, csv etc) using python. I can suggest you python and fast algorithm. It is no problem at all to do your job. Please send me a message to discuss more. Ki More

$80 USD in 3 days
(1 Review)
0.6

26 freelancers are bidding on average $195 for this job

leemilun

Thank you for your detailed project description. Very. Surely I can help you. I feel confidence all the requirements you want. I will do my best to satisfy you. Let's have a quick chat to discuss it further. Thanks aga More

$200 USD in 2 days
(49 Reviews)
6.3
DaniilSpasov

Hello. How are you today? I've read your job post carefully. This job really attracts me and also your required skills are completely matched with my major skills. I have good knowledge and rich experience with it. I b More

$500 USD in 10 days
(11 Reviews)
5.6
wonwon424

Hi, employer. I've read your proposals carefully and I think I can do it with python I have many previous works in this work and I will complete your project definitely. The person who waits too long loses the opport More

$150 USD in 3 days
(24 Reviews)
5.7
anuyadav1

I can write fast and efficient python script to analyse these files and save results after processing files as described .

$200 USD in 7 days
(62 Reviews)
5.9
adampohp79

Hello, I read out your task completely and I am interested in your post. I have confident that I will provide you quality work within your deadline. I have a deep passion for research and I guarantee that all of my wo More

$140 USD in 7 days
(54 Reviews)
5.8
$140 USD in 3 days
(44 Reviews)
5.0
KimHyokJun

Hi, glad to bid on your project. I have read your description and convinced I am suitable for your project. I have skills of C#/Java/Python in db management and programming related to db. I have worked 7+ years in DBMS More

$100 USD in 5 days
(16 Reviews)
4.6
albertpopov46

I am a Python expert. I have worked in Python development for 7+ years. And then I am a good english speaker. So anytime,anywhere you will discuss anything with me freely. Credit is the most important to me. No credit, More

$130 USD in 4 days
(14 Reviews)
4.4
BestService222

Hi, How are you? I've carefully read your description and I am very interested in your project. I am ready for your project, I can start immediately and finish it within the agreed deadline. I have expertise with Pytho More

$140 USD in 7 days
(23 Reviews)
4.7
mirzayas

hi sir can i help you to clean your data according to your desire and your requirement i have very good expreince in data cleaning.

$140 USD in 7 days
(7 Reviews)
4.5
DianaAli0608

Hello dear, I read your job posting carefully and am very interest your project. I am a full stack developer with great experience and skills in Web development for over 5 years long. I've always been giving clients sa More

$250 USD in 7 days
(13 Reviews)
4.3
Victorier006

hello,how are you. i read your bid carefully. i am Python, SQL expert and have full experience for 7 years. Python, SQL is my top skill and i can handle your project fully. I can provide most quality and high fast. if More

$150 USD in 7 days
(2 Reviews)
4.1
hectorenavarrou

Hi, I have a lot of experience optimizing databases to makes searches quicker as well as filling databases faster. I have lots of tips which can make the whole process a lot faster

$240 USD in 7 days
(9 Reviews)
3.9
Molotova

Dear sir. How are you? I have gone through the requirement and would like to raise my interest in the same. I am an expert at : Python, PHP, Mysql, Laravel, Codeigniter,JavaScript, jQuery etc. I believe I have most of More

$200 USD in 7 days
(4 Reviews)
3.2
greenguru2018

Hello How are you? I've read carefully your job description. I have more than five years experience in Development. Your satisfaction with the project is my top priority! If you give me a chance to work with you, the More

$250 USD in 7 days
(4 Reviews)
3.4
leeshuntie87

Need Web Development or Business Growing? My experiences range from building responsive software design that work well cross-browser to Business Goal. I am glad to provide full solution to you. We Are in House of Web a More

$400 USD in 7 days
(3 Reviews)
3.4
phantom1222

Hi I have read your job description carefully and I am very interested. I am a Python expert. And I have good experience in MySQL. I have worked in Python development for 7+ years. And then I am a good english speaker More

$140 USD in 7 days
(3 Reviews)
3.3
blackwhite123

Hello, I am pleasure with your job for Matching System. Thank you for the job posting. It’s a pleasure to meet you. I’d really like to work with you on this one if possible! I do have a couple of questions, but first I More

$140 USD in 7 days
(1 Review)
3.0
$155 USD in 3 days
(1 Review)
3.0
romansokolv708

Hi, I'm glad to bid your project. I am very interested in your job. I am a PHP&JS expert and confident on your job. I can do that with my this price for your successful product. As a senior web app full time develope More

$100 USD in 7 days
(3 Reviews)
2.8