Etl jobs

...current on-premises Qlik Enterprise Edition to a cloud-based solution. Here is an overview of the project scope: Migration of Local Apps to the Cloud with Enhancements: Transfer existing Qlik Sense applications from the local environment to the cloud. Implement any necessary enhancements or optimizations during the migration process. Migration of On-Prem ETL to Automation on Cloud: Migrate our existing on-premises Extract, Transform, Load (ETL) processes to automated workflows on the cloud platform. Ensure seamless data integration and transformation during the migration. Set up data sources: Move data sources from local files to Azure blob storage or Azure SQL databases on the cloud. Establish robust and efficient data transfer mechanisms between Azure cloud services ...

Job Description: Senior ETL QA We are seeking a skilled and experienced Senior ETL QA (Extract, Transform, Load Quality Assurance) professional to join our team. As a Senior ETL QA, you will play a vital role in ensuring the accuracy, integrity, and reliability of our data integration processes. You will collaborate closely with cross-functional teams to design and execute comprehensive testing strategies, identify and resolve defects, and optimize ETL workflows. The ideal candidate will have a strong background in data validation, ETL testing methodologies, and a passion for delivering high-quality data-driven solutions. Responsibilities: 1. Develop and implement comprehensive test strategies and plans for ETL processes, ensuring end-to-end data q...

Job Description: Database Engineer (Informatica/ETL) We are seeking a highly skilled and motivated Database Engineer with expertise in Informatica and ETL (Extract, Transform, Load) processes to join our dynamic team. As a Database Engineer, you will play a crucial role in designing, implementing, and managing our complex database systems, ensuring optimal data integrity, performance, and security. You will collaborate closely with cross-functional teams to deliver scalable solutions and contribute to the success of our organization. Responsibilities: • Design, develop, and implement efficient and scalable database solutions using Informatica PowerCenter and other ETL tools. • Perform data extraction, transformation, and loading (ETL) processes, ensuri...

Azure ADF Data Processing Project I am looking for a freelancer who is experienced in Azure ADF data processing. The project involves processing data from a source, such as SQL Server to another SQL server with some ETL. Ideal Skills:

Add history functionality to existing ETL process in pyspark. Need to account for racing conditions on primary key.

I am looking for a skilled professional who can help me with my project titled "Informatica Cloud Services ETL jobs". I need someone who has experience in working with Relational Databases and can perform Aggregation transformations on the data. The extracted data needs to be loaded into a Data Warehouse. Ideal Skills and Experience: - Experience with Informatica Cloud Services - Knowledge of Relational Databases - Expertise in Aggregation transformations - Experience in loading data into a Data Warehouse If you possess the required skills and experience, please contact me to discuss this project further.

Please Sign Up or Login to see details.

we are currently introducing an electronic technical logbook solution for all sorts of aviation sectors called REDiFLY. REDiFLY currently has a number of fleets under her belt in Europe and the middle east making it EASA certified and on its way to be GACA certified as well. we are looking for someone in the states to communicate with the airlines there and showcase our ele...and on its way to be GACA certified as well. we are looking for someone in the states to communicate with the airlines there and showcase our electronic tech log therefore making it known and used in the U.S aviation market. this individual will not only be paid for the job but also get a percentage of the sale for each deal he or she makes. in the attached files are more information about our ETL. website:

I am looking for a data engineer resume that showcases mid-level experience, with a focus on the finance industry. The ideal candidate should have experience in data warehousing, with a focus on projects related to data warehousing. Skills required include expertise in SQL, ETL tools, database design, and data modeling. The candidate should have a strong understanding of data warehousing concepts and be able to work collaboratively with business stakeholders and IT teams. Additionally, experience with cloud-based data warehousing solutions is a plus. The final product should be a visually appealing and well-organized resume showcasing relevant experience and skills.

... Justify the answer. c) Define the types of dimensions you have identified. d) For defined tables, fill them with some dummy data to more clearly reflect the implementation details. (eg 5-6 lines). 4. Explain how you will perform initial loading and delta loading. at least 2 possible transformations and give an example. (eg tables with relevant attributes and some data to explain the ETL process) Part 2: Tech Stack Determine the technological components (tools) needed to implement your solution. Analyze and justify your choice, including possible costs and the ability to adapt to future changes. Part of the assessment are: Provide examples of tables (attributes and related data) for points in the functional analysis section. Ability to present the paper within the specified t...

Hi, If you are looking for freelancer, I'm here for waiting for you! Please feel free to connect. About me: - Experienced Java Developer with 5 years of expertise in Spring-boot (Rest, Hibernate, Metrics,..), Java SE Java 8 & 11 (collections, multi thre...bus, IBM MQ) - SQL (POSTGRESQL, Oracle,...) and NO SQL (Couchbase, Mongo), Snowflake DB - OAUTH 2.0 for Azure and Google services - Selenium - Random fake data generation - API Testing (Rest, Soap) - Cloud services: AWS (S3, Dynamo DB, Cloud watch events, Lambda, Amazon MQ) and Azure (COSMOS DB ) - 3rd party/functional applications Automation - Notification system (Slack integration, email,..) - ETL testing for various databases, files including flat files, zip files, excel, JSON, XML, PDF using Apache spark, POI, PDF BOX...

I am seeking a skilled data engineer to help with my ETL pipeline and data analytics dashboard project. The ideal candidate should possess the following skills and experience: - Strong knowledge of SQL databases - Experience with ETL pipelines and data cleaning - Proficiency with data visualization tools - Ability to create a data analytics dashboard with the goal of visualizing data trends I am looking to build a real time analytics/visualization dashboard and would like to learn about how to set up ETL pipelines. The dashboard can be on anything using some, if not all of the technologies listed on the attached file. If there is a similar project that you have done in the past - that is fine, the focus here is that I would like to learn step by step on how to set up...

Ищем front-end специалиста, с хорошими навыками и опытом Vue 3 на part-time занятость (ориентировочно 10-15ч/нед, по мере поступления новых задач) в логистический стартап. Русскоязычная команда, переписка и созвоны на русском языке. Оплата почасовая, укажите свой рейт и ссылку на портфолио. Tech stack приложения: Product building: •Microservices architecture. Core api building: •Golang. ETL pipelines: •Python •Golang User interface: •Vue3 •Web socket RPC for real-time data and REST for communication with API Geographic data and routing: •OpenStreetMap Base database: •Postgresql •PostGis Messaging between components: •High performance distributed system NATS

Please Sign Up or Login to see details.

I am looking for an experienced ETL/Informatica developer to help me with a data warehousing project. The project involves moderate complexity level of data sources and needs to be completed within 1-3 months. The ideal candidate should have the following skills and experience: Skills: - Strong knowledge of ETL and data warehousing concepts - Experience in Informatica PowerCenter - Ability to design and develop ETL mappings, workflows, and sessions - Expertise in SQL and relational databases Experience: - Minimum of 2 years of experience in ETL development - Previous experience in data warehousing projects - Experience in working with moderate complexity level of data sources If you have the required skills and experience, please submit your proposal with you...

...занятость ориентировочно 10ч/нед. Обязательно наличие опыта с WhatsApp и Telegram API integrations. Если есть опыт в Golang и Vue3, то рассмотрим на долгосрочное part-time сотрудничество. Есть еще несколько full-stack задач. О нас: Логистический стартап с украинскими корнями для рынка США. Tech stack приложения: Product building: •Microservices architecture. Core api building: •Golang. ETL pipelines: •Python •Golang User interface: •Vue3 •Web socket RPC for real-time data and REST for communication with API Geographic data and routing: •OpenStreetMap Base database: •Postgresql •PostGis Messaging between components: •High performance distributed system NATS

...1. Strong experience in Python programming language 2. Experience in working with PostgreSQL databases and SQLAlchemy 3. Familiarity with CSV file handling and data manipulation 4. Experience in working with large datasets and optimizing performance 5. Strong problem-solving skills and attention to detail 6. Excellent communication skills Preferred Skills: 1. Experience in data engineering and ETL processes 3. Knowledge of best practices in database design and optimization Project Timeline: The project should be completed within 4-6 days, starting from the date of the contract. The timeline can be adjusted based on the project scope and complexity. Communication: The developer will be required to provide regular updates on the project progress and communicate any issues or ...

The project involves building a web application that provides a unique solution to a specific industry, combining elements of AI-powered insights, dynamic data visualization, and integrated data sources. The application will feature a chat interface for user interaction and will require skills in ETL, NLP language processing , Node.js, , and PostgreSQL. To respect the confidential nature of this project, I would require a signed Non-Disclosure Agreement (NDA) before we delve deeper into the specifics. However, I can assure you that this project offers an opportunity to work on cutting-edge technologies and presents considerable room for creativity and innovation.

Hello I need a Tableau developer to hire for an hour,i would provide a dataset in excel of 2.7mb and do the task with a screenshot in the report you write as required .No etl is need on the data is it clean

I am seeking a skilled database developer who can create a NoSQL database for analytics purposes. The ideal candidate will have experience with Cassandra, as this is our preferred technology. Key requirements for the project include: - Developing a NoSQL databas...candidate will have experience with Cassandra, as this is our preferred technology. Key requirements for the project include: - Developing a NoSQL database for analytics purposes - Expertise in Cassandra and other NoSQL technologies - Ability to design and implement an efficient and scalable database - Strong understanding of data modeling and architecture - Familiarity with data warehousing and ETL processes If you have experience in these areas and can deliver a robust and reliable NoSQL database, please apply for thi...

...support and guidance on data engineering best practices - Collaborate with our team to develop and maintain data pipelines, ETL processes, and data storage solutions - Optimize and scale data infrastructure to ensure high performance and reliability - Debug and resolve data-related issues, and provide recommendations for improvements Skills Required: - Strong experience with Python for data engineering tasks - In-depth knowledge of AWS services, including EC2, S3, RDS, and Kinesis - Hands-on experience with Apache Kafka for real-time data streaming and processing - Proficiency in using Snowflake as a cloud-based data warehouse solution - Familiarity with data modeling, ETL processes, and data warehousing concepts - Excellent problem-solving skills and the ability to work ...

Need original datasets including the end product for - 1 PowerApps Application - 2 PowerAutomate flow - 3 Power BI Dashboards (HRM, Financial Performance and any other interes...product for - 1 PowerApps Application - 2 PowerAutomate flow - 3 Power BI Dashboards (HRM, Financial Performance and any other interesting insights) No need for customized data. Use what you have. Need proper README file afterward. This is to show skill (I've done several of these but time does not allow it today) Objective: Portfolio for a beginner/intermediate Business Analyst. Main task: Show ETL to end product and show integrations abilities between Power Applications and other applications. And Develop productivity flow for Power Automate. Emphasis in replicating what is used in Business. ...

I want to build an ETL pipeline using the Stocks data from Kaggle. Please read the attached document in its entirety to understand the requirements properly. I have worked on the project and have attached my files (ZIP file) and needs to be edited according to the requirements.

I am looking to hire a Freelancer to develop an SQL SSIS SSRS project. The project will involve using data from SQL Server databases as data sources. Additionally, I need the project to have ETL (Extract, Transform, Load) capabilities and reporting or analytics capabilities.

We need a short study of how to change from infobright database to another columnar database. We are using the (deprecated) communiity version of infobright. We have mysql as source and kettle as ETL. - Which s better to move to and why - How to move - Proof of concept Req - SQL based queries on the new one We need an experienced freelancer. Needs to have a good knowledge on column-oriented databases.

...candidates' knowledge of data warehousing, ETL processes, data modeling, and data architecture. ? Analyze problem-solving abilities and aptitude for handling complex data engineering challenges. ? Collaborate with our hiring team to understand role requirements and align interview criteria. ? Provide detailed feedback and recommendations on candidates' technical abilities and cultural fit. Required Technical Skills: ✅ Strong proficiency in Apache Kafka and its core concepts, such as producers, consumers, topics, and partitions. ✅ In-depth knowledge of Apache Spark and its various components, including Spark Core, Spark SQL, and Spark Streaming. ✅ Familiarity with data warehousing concepts and technologies, such as dimensional modeling and OLAP. ✅ Expertise in ETL...

...UK-EU team remotely (but, preferably from within the UK-EU region). The ideal candidate would have experience in selling either : 1. Cyber security services, including Vulnerability Assessment & Penetration Testing, Network Security Testing, Cloud Security Audits, Security Compliance Assurance among others. 2. Big Data Testing Services, including Data Migration Testing, Data Warehouse Testing, ETL Testing and Validation, Business Intelligence Reports Testing among others. We are searching for a partner who can be responsible for our business development of our cyber security and big data testing services and solutions to potential clients in the UK and EU regions. Additionally, the partner should be capable of selling our services across an Europe-wide region. If you th...

We need Data Engineer ETL and reporting or Talend experienced developer for a part time. We will give 25-28k per month

We need ETL and reporting experienced developer for a part time. We will give 20-25k per month

I am looking for a SQL SSIS ETL expert who can fix a package execution failure error that I am experiencing. I am not sure if there are any custom components involved, but my SSIS version is SSIS 2016 or later. The ideal candidate should have: - Extensive experience in troubleshooting and resolving package execution failure errors - Proficiency in SSIS 2016 or later - Knowledge of custom components and their potential impact on ETL processes.

...solution is more at a conceptual level 5. Please address the below areas  Identifying the business requirements and existing issues in data operations (explain techniques used collecting requirements)  Data management operations relating to the various kinds of data that the company deals with.  Data Architecture (provide example of a proposed architecture that will help in processing the data e.g. ETL(datawarehousing or cloud solution)  Data quality measures  Metadatamanagement  Handling legacy data - Data migration  Data archival  Data governance measures  Data privacy  Expectedbenefits 6. The areas listed above are indicative and are in no sequence. When addressing this in the solution, please ensure you write in an orderly fashion. Also, any other data management ar...

...the reports and business logic. • Training end users in the use of reports and report production software. • Assuring the integrity of data, including data extraction, storage, manipulation and processing. • Data Model design and development, using SQL Server BI Stack & Power BI. • Data Warehouse Development, Maintenance & Support highly desirable. Technical Skill Set: • Data Warehousing, ETL, ELT, Data Modelling, OLAP, AI, Machine Learning, Forecasting, Predictive Analytics, Data Mining, Database Security and Version Control. • Microsoft SQL Server 2000 – 2021, Microsoft BI Stack, SSIS, SSRS & SSAS • Report visualisation, Report & Dashboard Development using some of the latest BI tools, SAP Business Objects, SSRS, ...

Project Scope: The project requires supply, install, configure, and build of automated data pipelines & near-real time dashboard service. The project design and development is for 6 months with total 12 months of support. Deliverables: 1. Supply and install the solution based on Alteryx and Power BI. 2. Develop BR (Business Requirement) & SD (Solution Design) documents based on use cases. 3. ETL (Extract, Transform & Load) data build an information product from desperate data sources using Alterx. 4. Build dashboards using Power BI. 5. Develop data profiling & prediction analysis via advanced analytical functions in both tools. 6. Complete knowledge transfer with detailed documentation & training of 5 Employees. 7. Provide support and maintenance services for 1...

...opportunities for improving integration processes and workflows. Requirements: Bachelor's degree in computer science, information technology, or a related field. Proven experience of 6-8 years in working with Workato, including designing and implementing integrations with Workday. Strong understanding of integration concepts and technologies, such as APIs, web services, REST, SOAP, JSON, XML, and ETL. Hands-on experience with Workato connectors, recipes, and workflows, and proficiency in designing complex integrations using Workato's visual interface. Solid understanding of Workday's data model, business processes, and integration framework. Strong problem-solving skills and ability to troubleshoot and resolve integration issues in a timely manner. Excellent commu...

I'm Muhammad Rizwan doing web development, building data pipelines, or deployment jobs as a Python developer. I've worked significantly with web scraping or data science-related tasks for the past 4+ years or more on or work with individual clients. I enjoy working on creative and fascinating projects and have a specialty in web development or in big data . Additionally, I have a...experience building prototypes and APIs. I have a lot of practical knowledge of the following technologies: Full Stack Developer | Angular framwork | React framwork | Django framwork | Big Data | Data Engineer | Data Scientist | DBMS | MySql | Cloud Computing | Python Programmer | web developer | Business Administration | Market Analyst | Data visualization | Data Wrangling | Data Warehousing | ETL...

Need to work on my project development work.. need informatica powercenter and IICS skills candidates.. skills-sql,etl,informatica,unix Need Work support for few months..charge monthly.. Required Telugu Candidates only..

Please Sign Up or Login to see details.

Need TALEND ETL developer to developer ETL jobs using TALEND ETL TOOL

...requirements for existing or new ETL pipelines. · Connect with stakeholders daily to discuss project progress and updates. · Work within an Agile process to deliver projects in a timely and efficient manner. · Design and develop Airflow DAGs to schedule and manage ETL workflows. · Transform SQL queries into Spark SQL code for ETL pipelines. · custom Python functions to handle data quality and validation. · Write PySpark scripts to process data and perform transformations. · Perform data validation and ensure data accuracy and completeness by creating automated tests and implementing data validation processes. · Run Spark jobs on AWS EMR clu...

Im having a Technology Training company, i need trainers to help me complete the class based on the instruction provided. Categories : 1. Data analyst - Excel and PowerBI 2. Data engineer - Database / ETL / Data mart building 3. Data scientist - Python / Prediction / optimization 4. Cloud Data Expert - Aws / Databricks / Hadoop So if you are housewife / trainer interested to take part time training in any of the below areas send me you resume with you monthly training quote - expected 6 hrs. per week of training.

Looking for Data Engineers having atleast 3 years of experience in end to ETL/ELT, data transformation. Candidate should have the following skillsets in AWS, Azure and GCP. Candidates having experience in any of the cloud platforms can also apply Only candidates who can work in US timezone (EST/CST) apply AWS services such as Glue, Lambda, Athena, S3, SNS, Kinesis, Data-Pipelines, Pyspark, etc. Kafka/Kafka Connect, Spark, Flink or AWS Kinesis Apache Nifi Dataflow Kubernetes AWS Data Pipeline Snowflake GCP tools - GCS, GKE, BigQuery, Cloud SQL, Cloud Connector Golang Airflow Typescript Data: DBT, Fivetran, Redshift, PostgreSQL Infra: GitHub, Bazel, Docker Azure Data Factory Azure Databricks DAX MDX Terraform Visualization tools knowledge - Data Studio, Amplitude,...

1. Java( Microservices , Web Api, Kafka, No SQL, Springboot)- Bangalore-Hybrid-8 Years 2. ASP.Net core Razor, Blazor, REST API, ETL, Kafka, RabbitMQ, C#.Net core, Redis Cache, MemCached, Mongo DB,-5+ Years-Bangalore 3. .Net with Angular-6-9 Years-NCR Hybrid 4. Data engineer-4-8 Years-NCR Hybrid 5. Python with some knowledge of React JS- Gurgaon- Hybrid 6. RoR- 2-3 weeks in Bangalore or Hybrid-5+ Years

hello i have a pws project for you - Create PS Library - Download XML/CSV/JSON via FTP/HTTP or otherwise - ETL the incoming data - Store in db (as JSON) - Extract valuable elements - Use on-demand resources

Please Sign Up or Login to see details.

A multi-billion auto-manufacturing company in the US is building a small team of top data-engineering and Salesforce experts to spearhead their marketing efforts. Very high-paced and dynamic environment, will require delivering immediate ...delivering immediate results. The immediate top priority is creating Marketing Cloud Journeys, Data Extensions through API, automation of monitoring and data validations, building new environments for SFMC. SFMC developer role requirements: Experienced Salesforce/SFMC developer/architect (min 5 years of experience) Recent hands-on expertise and deep knowledge of SFMC APIs. Expert in integrations, ETL, orchestrations, data modeling. Python, scripting, automations. Fluent in English A candidate needs to be available at least 5 hours between 8am an...

...meetings, and providing support to the team. This is an urgent position, and if the candidate is exceptional, we can onboard them quickly. Responsibilities: ● Lead a team of engineers in developing and maintaining Python-based applications ● Design and implement APIs and microservices ● Develop, maintain, and optimize SQL queries and stored procedures ● Utilize Snowflake for data warehousing and ETL processes ● Manage AWS infrastructure, including Lambda and Glue ● Work closely with stakeholders to gather requirements and provide updates on project status ● Provide technical guidance and support to the team ● Mentor team members to help them grow and develop their skills ● Collaborate with other teams to ensure successful project delivery Requirements: ● 6+ years of experience...

...are expected to do: - Design and build parts of a back-end trading system for crypto (Go); - Own the development process for these solutions as well as testing and rollout strategies, track all progress with different risks; - Work closely with React frontend developers to present trading system features; - Research and simulate trading strategies for users who plan to invest (Go and Python); - ETL crypto market data for historical analysis and statistics; - Use Redis+Postgresql for storing accounting, analytics and other trading related data; - Work on increasing the code testability and enforce the quality in our code. The Skills and Experience You Bring: - Excellent knowledge of the Golang and Python languages; - Good knowledge of JS, React, Typescript; - Experience in Redi...

...collaborate effectively across global teams · Independent thinker, willing to engage, challenge and learn new technologies. · Understanding of the benefits of data warehousing, data architecture, data quality processes, data warehousing design and implementation, table structure, fact and dimension tables, logical and physical database design, data modelling, reporting process metadata, and ETL processes. · Client facing skills: Solid experience working with clients directly, to be able to build trusted relationships with stakeholders...

My connection between Firebase Events data into BiGQuery was working fine but suddenly I am seeing data discrepancy. When I check the data on GA, I see correct data but when I look at the BigQuery analytics table, I don't see correct data. Sometimes data is correct, but mostly it's not. Need help from an expert who can look into this and suggest a solution.

...meetings, and providing support to the team. This is an urgent position, and if the candidate is exceptional, we can onboard them quickly. Responsibilities: ● Lead a team of engineers in developing and maintaining Python-based applications ● Design and implement APIs and microservices ● Develop, maintain, and optimize SQL queries and stored procedures ● Utilize Snowflake for data warehousing and ETL processes ● Manage AWS infrastructure, including Lambda and Glue ● Work closely with stakeholders to gather requirements and provide updates on project status ● Provide technical guidance and support to the team ● Mentor team members to help them grow and develop their skills ● Collaborate with other teams to ensure successful project delivery Requirements: ● 6+ years of experience...