Data ingestion and processing from Pub/sub to Bigquery

I'm trying to automate the ingestion of data from Pub/sub to Bigquery. I need help writing a cloud function to subscribe to a pub/sub, do some processing on that data to apply a certain logic, and then insert the data into Bigquery. I intend to use cloud scheduler, cloud pub/sub, cloud functions and big query. Do not intend to use apache beam as the volume of data is very low. (<100 records per day)


Complete writing cloud function which does the following:

Picks up data from pub/sub.

Processes the data and compares it to data in another table in BigQuery.

Based on certain conditions and criteria, the cloud function will insert that record into a separate table with an additional column which describes the status as 'Met', 'Not Met', 'Partially Met'

Skills: Python, Google Cloud Platform

See more: data entry processing officer, excel data entry processing controls, data entrydata processing, data entry processing skills, simply data email processing, scarborough data entry processing, data entry processing copywriting proof reading, data entry processing proof reading copy writing, copywriting proofreaders data entry processing, data entry processing copywriting proofreading rewriting, data entry processing copywriting proofreading engineering, data entry processing copywriting proof reading rewriting engineering, data entry processing copy writing proofreadingrewriting website design, vast experience data images processing, data encoding processing, I\ m trying to do data entry, data being published to pub/sub must be sent as a bytestring., data being published to pub sub must be sent as a bytestring

About the Employer:
( 0 reviews ) Italy

Project ID: #30955257

10 freelancers are bidding on average €249 for this job

(31 Reviews)

Hello, I wish you the best of luck in everything with you. As a professional developer, I have many years experiences of python Please feel free to contact me and let’s discuss about your job in detail to finish you More

€140 EUR in 7 days
(7 Reviews)

Hello, I am an experienced IoT developer & I have done the same project recently. I can start on this immediately. Thank you.

€1000 EUR in 15 days
(2 Reviews)

Hello Python EXPERT I have read your description and I am so interested in your project. You can see well experienced and skillful Python +5 years of experience in software development. Confident in your project and I More

€140 EUR in 7 days
(6 Reviews)
(4 Reviews)

Hello. I read through the job details extremely carefully. I am an expert in data scraping, data mining and data hunting, data collection with a great web search as well as having great research skills and tools. I am More

€140 EUR in 2 days
(1 Review)

Hello Sir, I am a software engineer and a professional content writer geared up with 8 years of experience under numerous niches. My writing umbrellas seo articles, blogs, academic writing, case studies and much more. More

€140 EUR in 7 days
(2 Reviews)

im pro in this cind of project you can test me if you want i already projects like this with successe

€180 EUR in 7 days
(0 Reviews)

Hello, I am a professional GCP Data Engineer with more than 13 years of experience. I do understand your requirement, and I have previous experience of working in such use cases. I can start working immediately. Let More

€105 EUR in 7 days
(0 Reviews)

Dear JobPoster, Introduction: I am Alok Ranjan M.Tech. in software systems from BITS Pilani with 17 years’ experience in the IT Industry. I am working in CMM Level 5 company as Technical Architect. Worked majorly in More

€145 EUR in 7 days
(0 Reviews)