Data ingestion and processing from Pub/sub to Bigquery

Closed Posted 2 years ago Paid on delivery
Closed Paid on delivery

I'm trying to automate the ingestion of data from Pub/sub to Bigquery. I need help writing a cloud function to subscribe to a pub/sub, do some processing on that data to apply a certain logic, and then insert the data into Bigquery. I intend to use cloud scheduler, cloud pub/sub, cloud functions and big query. Do not intend to use apache beam as the volume of data is very low. (<100 records per day)

Deliverables

Complete writing cloud function which does the following:

Picks up data from pub/sub.

Processes the data and compares it to data in another table in BigQuery.

Based on certain conditions and criteria, the cloud function will insert that record into a separate table with an additional column which describes the status as 'Met', 'Not Met', 'Partially Met'

Python Google Cloud Platform

Project ID: #30955257

About the project

10 proposals Remote project Active 2 years ago

10 freelancers are bidding on average €249 for this job

normanburtonfree

Hello, I wish you the best of luck in everything with you. As a professional developer, I have many years experiences of python Please feel free to contact me and let’s discuss about your job in detail to finish you More

€140 EUR in 7 days
(7 Reviews)
3.7
worksoft

Hello, I am an experienced IoT developer & I have done the same project recently. I can start on this immediately. Thank you.

€1000 EUR in 15 days
(2 Reviews)
3.6
AleksandarDikic

Hello Python EXPERT I have read your description and I am so interested in your project. You can see well experienced and skillful Python +5 years of experience in software development. Confident in your project and I More

€140 EUR in 7 days
(6 Reviews)
3.2
olgaklychkova194

Hello. I read through the job details extremely carefully. I am an expert in data scraping, data mining and data hunting, data collection with a great web search as well as having great research skills and tools. I am More

€140 EUR in 2 days
(1 Review)
1.6
SpotContent

Hello Sir, I am a software engineer and a professional content writer geared up with 8 years of experience under numerous niches. My writing umbrellas seo articles, blogs, academic writing, case studies and much more. More

€140 EUR in 7 days
(2 Reviews)
2.3
TheTeaTher

im pro in this cind of project you can test me if you want i already projects like this with successe

€180 EUR in 7 days
(0 Reviews)
0.0
sanutopia

Hello, I am a professional GCP Data Engineer with more than 13 years of experience. I do understand your requirement, and I have previous experience of working in such use cases. I can start working immediately. Let More

€105 EUR in 7 days
(0 Reviews)
0.0
asr56bee1cac570f

Dear JobPoster, Introduction: I am Alok Ranjan M.Tech. in software systems from BITS Pilani with 17 years’ experience in the IT Industry. I am working in CMM Level 5 company as Technical Architect. Worked majorly in More

€145 EUR in 7 days
(0 Reviews)
0.0