Closed

Stream a large json file s3 into postgres from java

I need help with streaming a large json file from s3 into a postgres table from java.

I have a working solution in python but I want to get it done using java

The basic idea is we get mutiple gzipped json files into our s3 from a third party provider.

I want to be able to process those files and load them into postgres without downloading the entire file into the memory

If we can get 1 file work that way we can then look into multithreading

The crux of it is read the content from gzipped json in s3 without downloading the entire file into the memory

More info: [login to view URL]

Skills: Java, J2EE, Software Architecture, PostgreSQL, AWS Lambda, JSON, PostgreSQL Administration, PostgreSQL Programming

See more: parse large json file java, split large json file online, split large json file python, split large json file, parse large json file python, php parse large json file, parse large json file android, read large json file, validate large json file, read large json file nodejs, parse large json file javascript, convert large json file to csv online, large json file react native, convert json file to string java, read large json file python, large json file example, python split large json file, split large json file java, search large json file, json file to object java

About the Employer:
( 2 reviews ) Austin, United States

Project ID: #32171658

40 freelancers are bidding on average $549 for this job

AwaisChaudhry

Hi, I have read the details I believe I can do this job. While I believe I have some queries which need to be clarified. For that I would request you to start the chat so we could clarify those. We can discuss the time More

$1650 USD in 30 days
(94 Reviews)
8.1
(191 Reviews)
7.8
ahmedbilal2506

Hi sir I am interested to work with you. I can provide you professional web design services. I can design for you according to your requirements. I will give you unlimited revision until you will not satisfy for your d More

$250 USD in 2 days
(75 Reviews)
5.6
ayesha0124

Hi there, How r u? I bid as i believe that I can help you with this project as i have the experience with PostgreSQL, Java, Software Architecture, AWS Lambda and J2EE. here is my portfolio: https://www.freelancer.com/u More

$750 USD in 28 days
(6 Reviews)
5.2
apgrl123

hi, i have rich experienced of java/javafx so that i can give you the perfect result. java/javafx is my top skill and i am majoring it for over 6 years. i have completed the several projects already so i have confiden More

$500 USD in 7 days
(10 Reviews)
5.3
oxankaorb

Hello, Thank you for your job posting. I have good experience to work with streaming large json file from s3 bucket in AWS. As a 5 yrs experience with software development, work with medium and big companies in russia. More

$500 USD in 7 days
(4 Reviews)
4.8
gauravgargcs

Dear sir, Hope you are doing well. I have read your project description, so I would like to work with you for the long term. I have 9 years of working experience as a front end web [login to view URL] 7 years plus of expe More

$500 USD in 7 days
(8 Reviews)
4.9
(21 Reviews)
4.6
(9 Reviews)
4.7
abdulmohsin

Happy to work on this assignment. I'm an expert in Java/JavaEE technologies having more than 14 years of experience, I believe in quality deliveries and providing free support ( unconditional) in case of any issues/ mi More

$500 USD in 7 days
(15 Reviews)
4.7
(9 Reviews)
4.8
(8 Reviews)
4.4
hsh564cf84accd96

We will do your Json work I am writing this proposal in order to work for you in Software and Web Development. We are highly trained professional developers seeking to freelance and earn online. Having a flair in prog More

$400 USD in 7 days
(10 Reviews)
4.5
Dutchstudent7750

Hi, Experienced with both Java and S3. My proposal is to stream the data from the files, and process the data line by line in Java using GZIPInputStream. Multiple lines may have to be kept in memory to recover the jso More

$250 USD in 7 days
(20 Reviews)
4.4
ivanovatale

I’m an IOI/APIO (International Olympiad in Informatics, Asia Pacific Olympiad in Informatics) medalist and ICPC (International Collegiate Programming Contest) semifinalist. I have 2 years experience in coaching informa More

$500 USD in 7 days
(3 Reviews)
4.3
progressivedott

Hi, I am interested in your project. I have looked at your requirement and understand your want to migration your solution from python to a Java solution. I am the right right guy for the job woth over 7 years experie More

$250 USD in 7 days
(5 Reviews)
4.2
RomanMatveev1

A Java Expert! Hello, dear client! Hope you are doing well. I have checked your requirements. Your project is not difficult for me at all. Please check my reviews to see my ability. Best regards for you!

$350 USD in 3 days
(11 Reviews)
3.9
gregtechnola9

Hello, Greetings for the day!!! I have gone through your requirement and understood every aspect of them. I can fulfill your requirement flawlessly. I have more than 4 years of experience with happy clients. I ass More

$750 USD in 30 days
(7 Reviews)
4.4
oleksdevr

Strong java and python are required skills for your project. I will do your task in really short period. Let us discuss this soon.

$750 USD in 7 days
(3 Reviews)
3.9
jovandevelopero1

hi, dear client I saw your project posted and I am very interested and confidentable. It's very suited for my skill and I will provide only good result. [login to view URL] [login to view URL] To gauge a More

$500 USD in 5 days
(1 Review)
3.2