Completed

Migrate Data from Staging Sql Table to ODS Table (MySql)

Requesting a query (or multiple queries) move data and transform it from the source (staging) to the destination (ODS).

Must use MySql 5.7.22 or earlier.

Destination table schema can be changed after review with me.

The 7z file contains the staging schema and lots of sample data. The sample data comes from 3 consecutive dates (see source_filename) column. You are also welcome to add more data or use your own data set. If you use your own data set, it must be provided with the solution.

While all staging data is in a single table for distribution, one or more days will be uploaded to staging, then your query run; more data will be uploaded to staging, your query will be run again. Staging will be truncated after each run.

Business Rules:

*revision_id is key for matching

*destination file_date must be extracted from staging source_filename column

*load_date = NOW()

*start_date, end_date indicate when the row was valid; see commentary below.

*Order of file load cannot be guaranteed (e.g. August 3rd may be loaded before August 1st)

*Multiple files may be loaded at once

Basic Happy Path Scenario:

Row is loaded to ODS from August 1st file: start date = 2020-08-01; end_date = default value (9999-12-31)

Same Row is loaded from August 2nd; this has a changed [login to view URL] row in ODS database is end dated (minus 1 ms); new row is inserted

Basic Out Of Order Scenario:

*August 2nd file is processed; row inserted; start_date = 2020-08-02

*August 1st file is processed; row exists and all data is the same. start_date should reset to 2020-08-01

*If prior rows exist that were end_dated, these need to be adjusted as well.

Multi File Happy Path Scenario:

Row is loaded to ODS from August 1st file: start date = 2020-08-01; end_date = default value (9999-12-31)

Files from August 2nd and August 3rd are loaded in one batch; same rules apply as the "Happy Path Scenario"

Multi File Out Of order:

*Could have August 2nd already loaded in the ODS, then August 1st and August 3rd are loaded

*Could have August 3rd already loaded in the ODS, then August 1st and 2nd are loaded.

Skills: MySQL, SQL, Database Programming, Data Processing, Data Warehousing

See more: data staging example, data warehouse staging best practices, how to create staging table in sql server 2012, mysql data migration different schema, ssis staging table example, how to create staging table in ssis, sql staging table example, etl staging tables, getafreelancercom grab web data mysql table, create mysql table form data, iphone collect data mysql table, convert xml data mysql table, mysql migrate data schema, lucene indexing data mysql table, migrate data sql server using, gathering data mysql table, retrieving data mysql table codeigniter, grab certain data mysql table, transfer data sql server mysql, export data mysql table word document php

About the Employer:
( 17 reviews ) Brooklyn, United States

Project ID: #26855241

Awarded to:

lordzephirot

Hi, I have read your job description carefully and i am very interested in this job. I am full stack developer and i am very familiar with Sql. I can start working right now and looking for a good long term relationsh More

$200 USD in 7 days
(16 Reviews)
4.9

11 freelancers are bidding on average $184 for this job

plikitik123

hello im a fullstack developer and also a software engineer i have read your description Requesting a query (or multiple queries) move data and transform it from the source (staging) to the but i need more detail via c More

$250 USD in 30 days
(5 Reviews)
3.5
bsgohel17

Hello, I hope you are doing well. I've gone through your project details and I'm confident we can deliver your requirements in a timely and professional manner. ♛ About Us -------------------------------- ✔ 8+ Years More

$140 USD in 28 days
(3 Reviews)
4.1
petruhno1

Just checked your job description very carefully including attached file. As i am passionate young web developer, i have strong knowledge & rich experience in database programming such as sql/mysql/nosql for over 7 yea More

$300 USD in 3 days
(2 Reviews)
1.0
designmaster786

Hello, Its very interesting to saw the description that based on the Migrate Data from Staging Sql Table to ODS Table (MySql) [login to view URL] [login to view URL] I have great HAND in all kind of More

$140 USD in 7 days
(0 Reviews)
0.0
Bisworanjanpradh

Hello,I'm highly interested in this project.I did same types of project [login to view URL] don’t be fell hesitate to invite me.I am expert in data entry, web search, web scraping, excel, copy typing [login to view URL] check my prof More

$135 USD in 7 days
(0 Reviews)
0.0
ak27freelancer

Hello, I am pleasure with your job for Migrate Data from Staging Sql Table to ODS Table (MySql). Thank you for the job posting. It’s a pleasure to meet you. I’d really like to work with you on this one if possible! I d More

$140 USD in 7 days
(0 Reviews)
0.0
innovativeteam

Hi, I've read and understood your job details thoroughly that you would like to move the data from the source to the destination using MySql 5.7.22. My name is Sabih and I am a Full-Stack web developer. I am an expe More

$140 USD in 7 days
(0 Reviews)
0.0
$100 USD in 5 days
(0 Reviews)
0.0
wahmad1983

Hi Team, Hope you are doing good. I am having around 11 years of IT experience which Includes mainly in the area of Database development,Design,Maintenance,Management along with worked on many data Warehouse projects. More

$278 USD in 15 days
(0 Reviews)
0.0
nobullmail

SENIOR WEB DEVELOPER ⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐ ================================================= ⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐ ================================================= ⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐ ================================================= ⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐ == More

$200 USD in 2 days
(0 Reviews)
0.4