Etl jobs

...glance how well we are moving product from warehouse to customer. The focus is delivery progress, so the visuals must surface three core metrics: on-time deliveries, route efficiency, and customer satisfaction scores. Data sources are already available in our internal SQL database and a daily CSV export from our last-mile partner. I can supply sample tables and field definitions; you will handle the ETL so the numbers refresh automatically each morning. A browser-based tool such as Power BI Service, Tableau Online, or a lightweight React-D3 build is fine—whichever you are fastest with. Key expectations • Single-page overview with KPI cards, trend lines, and a map or heat-grid for route efficiency • Filters for date range, region, and driver • Drill-do...

I already have Zoho DataPrep up and connected to my Zoho One account. The legacy records are largely well-structured, yet they still carry old duplicates, stray values, and obsolete pick-lists that must be scrubbed before the switch to m...the DataPrep workspace plus a copy of the target CRM field map. When we’re done I expect: 1. An import-ready dataset free of duplicates or formatting issues. 2. The saved DataPrep recipe so I can rerun it if new data arrives. 3. A brief hand-off note summarising any assumptions, formulas, or data quality flags. If you have real-world experience cleaning data inside Zoho DataPrep—or similar ETL tools such as Talend, Power Query, or Informatica—this should be straightforward. Please reference past Zoho DataPrep work so I know...

I’m looking for someone who has already moved Microsoft SSIS workloads into Google Cloud and can guide me through a clean, production-ready rebuild of our existing pipelines. Right now the jobs pull from a mix of relational databases and nightly flat-file drops; the goal is to land everything in BigQuery with the same—or better—performance, data quality, and monitoring we have today. Here’s what I need from you: • Assess the current SSIS packages, map out every source table and file feed, and document the equivalent flow on GCP. • Design and build the new pipelines using native services (Dataflow, Cloud Data Fusion, Cloud Composer, or any combination you feel is best) with full error handling, logging, and retry logic. • Automate scheduling and...

I’m preparing a migration from our Sapiens ID3 platform into the FAST Technology system and need a precise field-by-field mapping created. The source data centers on policy records, and I must be sure that every element tied to the customer—name and contact details, policy-specific attributes, and related transaction history—lands in th...transformation or new target field is required. Deliverables • A comprehensive mapping document (spreadsheet or equivalent) showing source field, data type, description, target field, and required transformations. • A brief summary of any assumptions or open questions that need business input. I’m looking for accuracy over volume: once I validate the mapping, we’ll move on to building the actual ETL, ...

...from multiple sheets or external CSV/XLSX files into a single, structured source of truth. 3. Automated reporting – generate dashboards in Google Sheets (or connected Looker Studio if you prefer) that refresh automatically and show at a glance: shipment delivery times, order-fulfillment rates, and current inventory levels. Deliverables: • One master sheet with validation logic in place • ETL or scripted connections for all incoming data sources • A fully formatted dashboard tab with the three KPIs above, filterable by date, destination, and carrier • Brief hand-off notes or a quick loom explaining how to maintain the setup If you’ve built logistics dashboards before and are comfortable with formulas, QUERY, ARRAYFORMULA, pivot tables, A...

...our treasury platform. I am comfortable sharing a sample dataset so you can map the fields, clean anomalies, and transform the numbers. Ideally the workflow is automated—SQL or Python for the heavy lifting and Excel, Power BI or Tableau for the final dashboards and printable summaries—but I am open to whichever stack you can justify as robust and maintainable. Acceptance criteria • A scripted ETL routine that I can run on new deal files without rewriting code. • Interactive dashboards plus a one-page PDF summary highlighting ROI, volume trends, and daily P&L. • Brief hand-off notes so my team can adjust data paths or field names in future. If you have questions about the data structure or need a quick walkthrough of the treasury trading ...

Title: Senior dMRV / Carbon Data Engineer (Verra + – Palm Oil / Biochar) Scope: We are an agentic + Web3 dMRV platform. We need a freelancer to assist prepare our data and calculations. Your tasks: Review historical data of POME SCADA exports. Map SCADA fields to Verra VCS POME/biogas methodology requirements and, at a high level, to biochar requirements. Build Python/SQL ETL scripts to clean and structure the data (handle gaps, outliers, units). Implement preliminary carbon calculations (CH₄ avoided / destroyed, EFB→biochar tCO₂e stored) using standard factors. Deliver a short technical memo (10–15 pages) describing data, assumptions, and methods that we can later refine with VVB. Must‑have experience: Hands‑on data work (Python + SQL) with time‑series /...

The Workday technical consultant will design and implement scalable Workday data ingestion pipelines and provide recommendations on best practices for extracting and integrating Workday data across hyperscaler environments (Azure, AWS, or GCP). Integration Approaches Under Evalu...(Workday Query Language) RaaS has been the traditional approach but is not preferred for future implementations. Key Requirements 5+ years of Workday technical experience (integrations, reporting, or data extraction). Proven experience implementing Workday data ingestion to cloud data platforms. Strong understanding of REST/SOAP APIs, RaaS, and WQL. Experience working with modern data platforms and ETL/ELT pipelines. Ability to recommend optimal ingestion strategies based on scalability, performance, and g...

I need an experienced AWS data engineer to design and build production-ready ETL pipelines in AWS Glue. The work centres on moving and transforming data from several source systems—relational databases (e.g., PostgreSQL, MySQL), NoSQL stores, real-time streams coming through Kafka/Kinesis, and a handful of internal/external REST APIs—into a clean, query-friendly layout in S3 and, ultimately, Redshift. Requirements: Strong AWS data engineering experiences with various aws services Experience building end-to-end data pipelines (schema discovery, ingestion, transformation, orchestration, monitoring) Experience working with relational databases like Oracle, MySQL, and SQL Server etc Experience with data ingestion from on-prem systems to cloud Experience with streaming pl...

...cleaning • remove duplicates • normalize titles • unify IDs 4. ETL pipeline • scripts to refresh the dataset monthly 5. Matching keys • imdb_id • tmdb_id Database Size Expectations Movies: 600k + TV Series 200k + Technical Requirements Preferred stack: • Python • Pandas • PostgreSQL or SQLite • API integration • ETL scripting Important Constraints Do NOT scrape IMDb pages. Use official datasets and APIs only. Goal of the Project The database will be used to: • compare against a personal movie rating history • calculate similarity between titles • generate predicted ratings • identify highly compatible unseen movies Accuracy of metadata is critical. Ideal Candidate Experie...

Custom ETL pipeline with postgress DB

...any schema changes made - Collaborate async with backend and product teams to align on timelines and minimize disruption You Bring: - Proven experience migrating large MySQL databases to PostgreSQL - Hands-on experience with Supabase (must-have) - Strong SQL skills across both MySQL and PostgreSQL dialects - Understanding of schema design, indexing, and relational data modeling - Experience with ETL pipelines and data transformation tooling - Ability to identify and resolve data quality issues during migration - Strong attention to detail and a zero-tolerance approach to data loss Bonus: - Experience with pgloader, AWS DMS, or similar migration tools - Familiarity with Row Level Security (RLS) in Supabase - Experience working within SaaS or product environments - Scripting expe...

...an accomplished Senior Data Engineer to sit in on our technical interviews for roughly two hours each day. The role is purely evaluative: you will craft probing questions, join live video calls, and quickly score each candidate’s depth of knowledge across Python, Scala and SQL. Our stack centres on Azure and Databricks, so practical insight into large-scale Spark/PySpark jobs, data-model design, ETL orchestration and cloud performance tuning is essential. Candidates frequently discuss streaming, optimisation strategies and modern AI/ML add-ons, so any hands-on exposure to libraries such as PyTorch, NumPy, SciPy or TensorFlow will help you challenge them at the right level, though it is not mandatory. Availability is limited to two focused hours per weekday; I will sha...

I need a compact proof-of-concept that automatically extracts Documents from an Azure Blob container, captures their accompanying metadata, and lands both in a SharePoint Online document library. Only files classed as “Documents” (e.g., PDF, DOCX, XLSX) are in scope—no images or videos. The PoC must demonstrate that file contents arrive intact, their metadata is mapped to the correct SharePoint columns, and the process can be triggered on demand or on a simple schedule. You are free to choose the most suitable approach—Azure Data Factory, Logic Apps, Power Automate, an Azure Function, or a lightweight script in C# or Python using the Azure SDK and Microsoft Graph/SharePoint REST API are all acceptable. The key is clarity and repeatability; hard-coded secrets or one...

...Responsibilities Automation Architecture: Design and deploy high-concurrency automation clusters using Node.js (TypeScript) and headless browser frameworks (Playwright/Puppeteer). Stateful Flow Simulation: Develop sophisticated scripts to simulate complex user journeys, including geo-localized session management and extraction of landed costs (Price + Tax + Freight). Neural Data Normalization: Architect ETL pipelines to transform unstructured web data into our Internal Universal Material Schema using advanced regex and LLM-assisted parsing. Perceptual Indexing: Implement pHash (Perceptual Hashing) logic to identify and de-duplicate identical physical products across the national market. Infrastructure Management: Manage auto-scaling containerized workloads on AWS (Fargate/EC...

...datasets. --- ## B. Predictive Models AI must provide: * Cash flow forecasting * Inventory restocking prediction * Revenue forecasting * Fraud risk detection * Expense anomaly detection * Tax anomaly detection * Slow-moving inventory alerts * Client financial risk scoring --- ## C. Architecture Requirements The developer must propose: * Separate AI microservice OR data warehouse model * ETL pipeline from client databases * Secure API-based data access * Scheduled model retraining * Explainable outputs * Dashboard integration into firm admin portal --- # 9. Security (Non-Negotiable) The system must follow zero-trust principles. ## Administrative Access * All admin access must be SSH-only via: * VPN or Zero-Trust Gateway * Bastion host * Key-based SSH authenticati...

...covers the entire pipeline: extracting data from the current MySQL instance, scrubbing and de-duplicating it, mapping old structures to the target schema, and applying significant transformations to fit new business rules. Solid SQL is obviously essential, and you should already feel at home with large tables, indexes, constraints, and performance tuning. Experience scripting in Python or using ETL utilities (e.g., Talend, Pentaho, custom shell/Python pipelines) will make life easier because several transformation rules are complex. I will provide full access to the existing database schema, sample data, and the target data model on day one. Throughout the month you will collaborate with our in-house QA team to validate row counts, referential integrity, and transformation accur...

The task centres on moving content that currently lives in Azure Blob Storage into two destinations—SharePoint Online and a third-party cloud repository that we will finalise together. The source consists of mixed-format files: Word documents, images, CSVs and a few other common types. I need a lightweight yet robust ETL solution that can: • automatically detect new or updated blobs, • perform any required transformations (basic metadata enrichment, naming conventions, optional compression where helpful), and • deliver the output to the target platforms with solid error handling and retry logic. An Azure-native approach—Azure Data Factory, Logic Apps, Functions, or a combination—will fit best, but I’m open to other suggestions so lo...

...Advanced filtering UI Data grid with pagination & lazy loading Infrastructure Kubernetes / Docker Load balancing CDN Caching (Redis) Object storage for raw data Required Features ================ Advanced Search Engine Full-text search Multi-filter query builder Auto-suggestions Fuzzy matching Aggregations (sum, count, trends) Data Handling ============ Bulk data ingestion pipelines ETL processing Schema optimization Index optimization Performance Requirements ====================== Query response in seconds Pagination with deep offset handling Parallel query execution Caching for repeated queries Security & Access =============== User authentication Role-based access Paid subscription model (optional phase 2) Dataset Details ============= 10+ TB...

I have several CSV and JSON files arriving on a regular schedule, and I need a reliable ETL pipeline that ingests these flat files, cleans the raw data, and validates every record before loading it into my environment. The core of the job is to: • Read multiple flat-file formats (mainly CSV, with the occasional JSON). • Apply thorough data-cleansing rules—removing duplicates, enforcing data types, flagging out-of-range values, and normalising text fields. • Run validation checks so that only clean, schema-compliant rows proceed to the load step. I’m happy for you to choose the stack you are most efficient with—Python (pandas, PySpark), Talend, or another ETL tool—as long as the final solution is reproducible and can be trigge...

I have direct access to our e-commerce sale...connections or scheduled refresh against the database. I’ll provide the connection string and a sample export; you’ll propose the data model, set up the transformations, and surface the insights in clean, intuitive visuals. Exact metrics and calculated fields can be finalised together once the framework is in place. Deliverables • Secure connection to the sales database, including any necessary ETL or query optimisation • Fully-interactive dashboard with filter panels, drill-through views, and export options • Clear hand-off documentation covering data model, refresh schedules, and how to extend the report Please outline the tool you prefer, the timeline you need, and one example of a similar dashboar...

For my data‐management course homework, I must build a complete ETL workflow with any free or open-source tool such as Talend Open Studio, Apache NiFi, or Pentaho. The task starts with extracting a small, publicly available sample data set (CSV, JSON, or a simple relational dump), then cleaning and transforming it—deduplicating records, resolving missing or inconsistent values, and normalising key fields where needed. Once the data is tidy, it has to be loaded twice: first into a staging target (a plain relational table or file storage) and then into a basic star- or snowflake-style data-warehouse schema so I can run simple analytical queries afterward. I will need the full project files, transformation jobs, and a concise write-up that walks through each step, explains the...

I want to spend one focused week mastering end-to-end data-engineering pipelines on Google Cloud. The heart of the bootcamp is hands-on practice: together we will build, deploy, monitor and troubleshoot real ETL flows until I can run them solo with confidence. Primary learning goal My priority is data ingestion, extraction, transformation and storage. We will start with the full ingestion toolkit—DataFlow, Pub/Sub and Cloud Storage—then chain that work into the wider GCP ecosystem: • Storage layers: BigQuery, Bigtable, Cloud Spanner, Cloud SQL, Datastore / Firestore • Transformation & orchestration: DataFusion, DataProc, Cloud Composer, Cloud Scheduler, Cloud Functions • Data quality & cataloging: DataPrep, Data Catalog • Visualisati...

looking for an experienced Azure Data Engineer to support and enhance our existing data platform on an ongoing basis. You should be strong in: Azure Data Factory (ADF) for building and maintaining ETL/ELT pipelines Azure Databricks and PySpark for large‑scale data processing Python for data engineering utilities, automation, and integration Delta Lakes/Lakehouse concepts, performance optimization, and troubleshooting Working with SQL‑based data sources, data warehousing, and BI integrations Responsibilities Design, build, and optimize data pipelines in Azure ADF and Databricks Develop and maintain PySpark and Python jobs for batch and near real‑time workloads Implement best practices for data quality, observability, and monitoring Collaborate with our internal team, follo...

...with every URL and a clear field dictionary so you can move straight to extraction with Python, BeautifulSoup, Scrapy, or whichever tool chain you prefer. The finished dataset must come back as a single Excel/CSV file. Before you hand it over, give it a quick polish: apply basic, uniform formatting, drop any duplicates, and make sure each column lines up with the field names I supply. No heavy ETL work—just that first-pass cleanup so I can analyse the file immediately. Deliverables • CSV (or XLSX) containing 13 columns × ~285 rows, fully populated where data exists • Basic formatting and de-duplication applied • Short note flagging any URLs or fields that could not be captured A quick turnaround is ideal; the job should be straightforward...

We are seeking an experienced Data Engineering & Data Analytics Specialist to support the design, development, and optimization of modern data pipelines and analytics solutions. - Scope of Work Design, build, and maintain scalable ETL/ELT data pipelines Develop and optimize data models for analytics and reporting Work with cloud data platforms (AWS, Azure, or GCP) Implement data validation, quality checks, and monitoring Build dashboards and actionable insights using BI tools Collaborate on performance tuning and data architecture improvements - Required Skills Strong experience with SQL and Python Expertise in modern data stack tools (e.g., Snowflake, BigQuery, Redshift, Databricks) Hands-on experience with orchestration tools (Airflow or similar) Experience with db...

I need a skilled data analyst to help with Extracting, Transforming, and Loading (ETL) financial data into Excel. The focus will be on revenue and expenses. Key Requirements: - Analyze financial data, specifically revenue and expenses - Structure data currently from external sources - Proficient in Excel and ETL processes Ideal Skills: - Experience with financial data analysis - Strong Excel skills - Familiarity with handling external data sources

...and sitting in our SQL-based data warehouse; the missing piece is a set of polished, interactive Power BI visuals that turn those rows into clear stories for management. Here is what I need: • Connect Power BI Desktop directly to the database (SQL credentials will be provided) and build a streamlined data model—relationships, calculated columns, and DAX measures where helpful, but no heavy ETL work. • Design visually consistent report pages that highlight our key metrics with slicers, drill-through, and tooltips so users can explore the data on their own. • Publish the finished .pbix to Power BI Service, set up a workspace, and configure refresh schedules. • Hand over the final .pbix, the published dashboard link, and a short walkthrough expl...

I’m preparing fo...after each assignment will keep me on track. Deliverables • A week-by-week study plan mapped to the exam objectives • Reusable Databricks notebooks and Snowflake scripts created during lessons • Practice questions or mini-projects for each topic and reviewed answers • Final mock assessment-style project that ties everything together Acceptance criteria I can independently build a simple ETL pipeline in Databricks, tune a Snowflake query to meet a stated SLA, explain the platform architectures, and score at least 80 % on the provided mock exam. When those boxes are ticked, I’ll know the tutoring engagement has succeeded. Please outline your proposed schedule, teaching approach, and any prior experience guiding students t...

I need a small, reliable system that polls the Upstox API once a minute for roughly 500 equities and captures only one metric—Market Cap. Each pull should be written straight i...• A single command spins up the database schema and scheduler • At least eight trading hours’ worth of data loads without a miss or duplicate row • Daily report matches manual spot checks of raw figures • Code passes a quick review for clarity, error handling, and PostgreSQL best practices If you have previous experience with market data feeds or the Upstox SDK that’s a bonus, but solid ETL skills and a knack for building resilient jobs are what really matter here. I would like to see the first working pull-and-store cycle within a few days so we can fine-tune ...

...future team members can follow your logic • Basic deployment notes (gateway setup, refresh schedule, role-level security) Acceptance criteria: the report refreshes without errors directly from the SQL source, renders each visual in under three seconds on typical desktop hardware, and is free of hard-coded file paths or credentials. If you have experience optimising DAX, using Power Query for ETL, and designing executive-ready dashboards, I’d love to see examples of your previous work and get started right away....

...entire ETL process, integrates with Snowflake for storage and transformation, and connects to Dun & Bradstreet (D&B) for enrichment and deduplication. This agent should not only process data but also understand it: identify quality issues, propose corrections, enrich missing details, and guide users through validation when needed. • Build a CSV ingestion pipeline with schema validation and error reporting. • Implement automated data profiling to detect missing values, inconsistencies, and duplicates. • Integrate AI models to generate cleansing suggestions and standardize formats. • Connect to D&B APIs for company lookup, D‑U‑N‑S retrieval, and firmographic enrichment. • Implement fuzzy matching and deduplication logic combining AI scorin...

...focuses on designing and implementing scalable, automated data pipelines and integration workflows within Oracle Cloud Infrastructure (OCI) to support Finance and Order-to-Cash (O2C) data flows. Key Responsibilities * Design, develop, and manage data integration pipelines using OCI Data Integration (OCI DI) for ingestion, transformation, and delivery of Finance/O2C data. * Build and orchestrate ETL/ELT workflows leveraging OCI services such as Object Storage, Data Flow, Data Catalog, Autonomous Data Warehouse (ADW), and Data Lakehouse. * Integrate data from Oracle Cloud ERP, legacy sources, and external applications into unified data models for analytics and reporting. * Implement data transformations, data quality checks, and reconciliation logic aligned with Finance business r...

...a clear, descriptive look at past inventory activity captured in our Oracle ERP system. The goal is a concise summary report that tells me, at a glance, how stock levels have moved, where turnover is strong or weak, and which items or locations stand out as anomalies. You will extract the relevant inventory tables (or work with a CSV extract I provide), shape the data with SQL or any preferred ETL approach, and present the findings in an executive-friendly format. Accuracy and interpretability are more important than advanced forecasting; this is strictly descriptive analysis. Core deliverables • Cleaned and consolidated dataset of historical inventory transactions • Executive summary report highlighting key metrics (stock movement, average holding days, turnover,...

...a clear, descriptive look at past inventory activity captured in our Oracle ERP system. The goal is a concise summary report that tells me, at a glance, how stock levels have moved, where turnover is strong or weak, and which items or locations stand out as anomalies. You will extract the relevant inventory tables (or work with a CSV extract I provide), shape the data with SQL or any preferred ETL approach, and present the findings in an executive-friendly format. Accuracy and interpretability are more important than advanced forecasting; this is strictly descriptive analysis. Core deliverables • Cleaned and consolidated dataset of historical inventory transactions • Executive summary report highlighting key metrics (stock movement, average holding days, turnover,...

...analysts, engineers, or BI specialists with strong communication and writing skills to contribute technical articles to our blog. Your mission? Share hands-on experience, real-world examples, and practical tips to help fellow data professionals work smarter. Who are we? ClicData is an all-in-one data management and business intelligence platform (SaaS), offering data connectivity, warehousing, ETL, data visualization, and automation. Our audience includes data professionals and data-savvy business leaders, primarily in mid-market companies across North America. Why write with us? - Were looking for long-term collaborators not one-off gigs. That means predictable, recurring income for you. - You'll be credited as the author of each piece you write your expertise will be sho...

...DataStage, QualityStage, or any complementary tools you recommend—so that golden records are created reliably and maintained automatically. What I expect from you • A short discovery session to understand the present architecture and pain points • A documented integration strategy that covers databases, cloud endpoints and REST/SOAP APIs • Detailed mapping specifications and any DataStage/ETL jobs or services needed to implement them • Configuration of matching, standardisation and survivorship rules inside MDM • Test plan plus before/after data-quality metrics to prove accuracy gains • Knowledge-transfer call so I can operate and extend the setup once you are done If you have demonstrable experience delivering clean, tr...

... PostgreSQL, MySQL, etc.) • Solid understanding of software engineering best practices and testing • Experience building and maintaining scalable full stack applications • Strong communication and team collaboration skills • Expertise with Python and various python packages Preferred Skills • Advanced SQL with PostgreSQL: joins, subqueries, CTEs, and query optimization • Understanding of ETL and Data ingestion • Experience with core AWS services: EC2/ECS, ELB, S3, CloudFront, IAM • Experience in Scala • Familiarity with microservices architecture and distributed systems • Knowledge of Terraform, Docker, Kubernetes • Experience with CI/CD pipelines and end-to-end testing tools like Cypress • Knowledge/Experie...

I need an experienced Python engineer who works confidently with AWS Glue to build and manage a small suite of data-integration jobs for a Hyderabad-based project. The core of the work is to design and automate Glue ETL pipelines that pull data from our production databases, catalog it accurately, and transform it into analytics-ready tables. Here is what I expect from the engagement: • Develop, test, and deploy Glue ETL jobs in Python. • Populate and maintain the Glue Data Catalog so new tables are discoverable and properly version-tracked. • Implement efficient transformation logic that cleans, enriches, and partitions data for downstream reporting. • Optimise job performance and cost by selecting the right worker types, job parameters, and data...

I need an experienced Python engineer who works confidently with AWS Glue to build and manage a small suite of data-integration jobs for a Hyderabad-based project. The core of the work is to design and automate Glue ETL pipelines that pull data from our production databases, catalog it accurately, and transform it into analytics-ready tables. Here is what I expect from the engagement: • Develop, test, and deploy Glue ETL jobs in Python. • Populate and maintain the Glue Data Catalog so new tables are discoverable and properly version-tracked. • Implement efficient transformation logic that cleans, enriches, and partitions data for downstream reporting. • Optimise job performance and cost by selecting the right worker types, job parameters, and data...

...AI-powered natural language query interface on top. ## Scope of Work **Phase 1: Data Integration to BigQuery** Connect and automate data pipelines from: - Google Analytics 4 (GA4) - Google Ads - Google Search Console - Google Tag Manager - Google Merchant Center Requirements: - Set up automated, near real-time data transfers - Design efficient BigQuery schema with proper data modeling - Implement ETL/ELT processes with data quality checks - Create unified views combining data across platforms - Import historical data - Document all data flows and transformation logic **Phase 2: AI Integration Layer** Implement AI-powered interface for natural language querying of BigQuery data. Requirements: - Configure secure connection between AI platform and BigQuery - Set up authentica...

I need an experienced Python engineer who works confidently with AWS Glue to build and manage a small suite of data-integration jobs for a Hyderabad-based project. The core of the work is to design and automate Glue ETL pipelines that pull data from our production databases, catalog it accurately, and transform it into analytics-ready tables. Here is what I expect from the engagement: • Develop, test, and deploy Glue ETL jobs in Python. • Populate and maintain the Glue Data Catalog so new tables are discoverable and properly version-tracked. • Implement efficient transformation logic that cleans, enriches, and partitions data for downstream reporting. • Optimise job performance and cost by selecting the right worker types, job parameters, and data...

I’m overhauling a Power Apps Canvas app and power query and want the interface to look and feel far slicker. The sole focus is a dashboard that surfaces our sales data in a way managers can grasp at a glance. That means: • Designing a responsive Canvas screen with charts, KPI cards and slicers that load fast on desktop and mobile • Shaping and scheduling the underlying data with Power Query so the numbers update automatically—no manual refreshes or copy-pasting • Leaving me with a short hand-over video or doc so I can tweak visuals or extend the query later Everything sits in SharePoint lists and an Excel workbook today; use whatever combination of Dataverse, collections or direct connectors keeps things fast and maintainable. Native Power Apps contro...

...unifying those streams into a clean, continuously updated dataset. On top of that data layer, the build must train, evaluate, and deploy the best-performing predictive models—whether regression, decision-tree, neural-network, or any other technique that proves superior—then surface the results through a lightweight web interface and an API our teams can call in real time. Key deliverables • Automated ETL jobs and data-quality checks for the three sources mentioned above • Modular training pipeline with experiment tracking, lift/ROC reporting, and feature-importance visuals • Scoring service exposed via REST (or GraphQL) endpoints plus an intuitive dashboard for non-technical users • Deployment scripts, environment setup notes, and a live han...

I have several disparate sources holding our inventory information and I want it all pulled together into one clean, well-structured Microsoft SQL Server database file. The job is straightforward in principle: extract every piece of inventory data you can reach, reconcile d...complete when I can: • restore or run the script on my Microsoft SQL Server instance without errors, • query a consolidated Items table and see one row per unique SKU, • join stock levels to their warehouse locations with no orphan records, and • export a sample CSV of current inventory that matches today’s live figures. T-SQL proficiency is essential; SSIS, Azure Data Studio, or any other ETL tooling you prefer is fine as long as the final deliverable is the ready-to-go SQL Se...

...through Power BI, so once the model is validated I’ll ask you to craft intuitive dashboards that highlight drivers, confidence ranges and any red-flag anomalies the model detects. Solid statistical grounding is essential; I want clear explanations of feature importance, assumptions and limitations that business stakeholders can grasp quickly. Big-data exposure, cloud familiarity (Azure, AWS or GCP), ETL pipeline design and MLOps practices are all welcome extras—you’ll have room to propose improvements if they make the solution more robust or scalable. Deliverables I need from you: • A well-documented predictive model with reproducible code and clear version control • Cleaned and transformed datasets stored back into SQL (or a recommended alternativ...

...social protection, and legal frameworks. Below are the positions and their respective qualifications: 1. Senior Project Manager - PMP or PRINCE2 certification - 10 years of experience managing complex projects in IT/e-government 2. Principal IT Architect - Degree in Computer Engineering or equivalent, TOGAF certification - 10 years of experience as an enterprise architect with expertise in API, ETL, cloud technologies 3. Business Architects/Analysts - Degree or Master’s in Computer Security, Data Science, or Computer Science - Expertise in implementing Digital Public Infrastructure (DPI) approaches 4. Cybersecurity and Data Security Expert - Degree or Master’s in Cybersecurity or IT Governance - 10 years of experience with CSIRT/SOC, PKI, IAM 5. Data Management E...

...recorded purchase prices Notifications or alerts for discrepancies or missing data Secure storage of credentials (where needed) and compliance with data protection/privacy guidelines Easy scalability as the number of suppliers increases Required skills/experience: API or data source integration (REST/SOAP, CSV/JSON/XML imports; web scraping only when permissible) Experience with data pipelines, ETL processes, scheduling (Cron, CI/CD tools, Zapier/Integromat or similar) Knowledge of price and stock data reconciliation, logging, error handling Handling secure authentication, token management, secrets management Ability to implement discrepancy alerts (email/Slack/chat alerts) and provide a simple dashboard or reporting interface What we offer: Clear list of suppliers with access ...

...auditability, integrity, and security 3. HIGH-LEVEL CLOUD ARCHITECTURE Core components: Network Layer AWS VPC (multi-AZ) Private subnets per regulated institution Central supervisory subnet Data Layer S3 Data Lake (Raw / Processed / Curated) Redshift / Aurora (analytics storage) Object Lock for integrity Compute & Processing Lambda (validation, rules engine) EC2 (stress testing engines, Monte Carlo) Glue ETL (transform pipelines) Step Functions (workflow orchestration) Streaming & APIs API Gateway (data submissions) Kinesis (real-time data ingestion) AI / ML SageMaker (fraud detection, early warning models) Neptune (graph AML network analytics) Redshift ML (ratio prediction) Monitoring & Security IAM / RBAC KMS encryption CloudTrail / GuardDuty CloudWatch Visualizati...

I need a reusable ETL framework built inside Databricks notebooks, version-controlled in Bitbucket and promoted automatically through a Bitbucket Pipeline. All source data arrives via GraphQL APIs, so the job includes handling authentication, pagination, and schema inference before landing raw payloads in Delta tables. A dedicated cleaning stage must then standardise and validate the data before it moves on to the curated layer. The structure should be modular—ideally a bronze/silver/gold notebook hierarchy—so I can slot in new sources or extra transformations without touching the core logic. I also want a lightweight Python package (wheel) that wraps the GraphQL connector and can be attached to any cluster. Acceptance criteria • Parameter-driven notebooks organ...