Hadoop jobs

Looking for a freelancer to help with a simple Hadoop SPARK task focusing on data visualization. The ideal candidate should have experience in: - Hadoop and SPARK - Data visualization tools and techniques - Ability to work quickly and deliver results as soon as possible. The task is: Use the following link to get the Dataset: 1- Using Hadoop SPARK software execute three examples: simple, moderate, and advanced over the chosen DS. 2- For each case write the code and screenshot for the output. 3- Visualize the results of each example with appropriate method.

...procorpsystem rn <<mail here >> rn Reply to:  << mail here >>rn rn rn rn rn rn rn rn rn Role:  Hadoop Developer / Admin with Production SupportLocation: Austin, TXDuration: 12 Months Job Description: We are looking someone having strong experience in production support, administration and Development experience with Hadoop technologies.• Minimum Experience 8 Years• Must have Hands-on experience on managing Multiple Hortonworks Clusters. Troubleshooting, Maintaining and Monitoring is the key responsibility here.• M...

...System Development Engineer to join our team for a remote project in Korea. The ideal candidate should have expertise in Hadoop, HDFS. Responsibilities: - Coding and implementing HDFS system development tasks - Collaborating with the team to design and develop efficient solutions - Conducting thorough testing and debugging to ensure system reliability [Project Overview] We are looking for skilled professionals capable of enhancing and developing the HDFS technology used in the NMS (Network Management System) related systems of one of the three major Korean telecommunication companies. [Detailed Job Description] 1. Establish HDFS cluster using the vanilla version of Hadoop - Expected integration with 16 servers 2. Build a Data Warehouse based on HDFS - Construct a sys...

I am seeking an experienced data engineer to perform training for my company. We are looking for somebody with advanced knowledge in Python, as well as Big Data technologies such as Hadoop and Spark. The training should last 1-2 months, teaching our team the fundamentals as well as more advanced applications of these skills in data engineering. If you think you have the experience and qualifications to provide this training, I encourage you to submit your proposal.

I am looking for a talented data scientist to help with a project that requires data analysis, machine learning and data visualization. I have medium-sized data sets ready to go, at between 1,000 and 10,000 rows. The data sets are...I'm seeking someone who can make sense of the data and use it to create data visualizations. This person should possess a strong understanding of machine learning and data analysis principles. The successful applicant will be expected to translate data into a visual form that will be easy to understand and communicate to others. Any experience with software such as Python, R, SPSS, Apache, and Hadoop will be greatly beneficial. If you think you have the skills to produce great results and get the job done, then please get in touch and let me know ho...

Quantori is a new company with a long history. We have over twenty years' experience in developing software for the pharmaceutical industry and driving advanced strategies in the world of Big Data revolution. ...- Good written and spoken English skills (upper-intermediate or higher) Nice to have: - Knowledge of web-based frameworks (Flask, Django, FastAPI) - Knowledge of and experience in working with Kubernetes - Experience in working with cloud automation and IaC provisioning tools (Terraform, CloudFormation, etc.) - Experience with Data Engineering / ETL Pipelines (Apache Airflow, Pandas, PySpark, Hadoop, etc.) - Good understanding of application architecture principles We offer: - Competitive compensation - Remote work - Flexible working hours - A team with an excellent...

I am looking...looking for an experienced Hadoop engineer to assist with troubleshooting and optimization of our existing Hadoop cluster. Ideally, the hiring candidate will need to demonstrate a high level of proficiency in relevant Hadoop technologies, as well as experience in troubleshooting and optimization. This engineer will be responsible for monitoring the performance of our Hadoop cluster, making adjustments and ensuring the environment is running efficiently. This individual must be able to identify possible inefficiencies and areas for improvement, providing solutions and suggestions for achieving better performance and scalability. This engineer should also be knowledgeable of data processing and analysis, as well as provide the necessary tech support...

Need to fix the missing files and blocks issue on AWS EMR cluster on has corrupt files/jars/blocks.

Quantori is a new company with a long history. We have over twenty years' experience in developing software for the pharmaceutical industry and driving advanced strategies in the world of Big Data revolution. ...- Good written and spoken English skills (upper-intermediate or higher) Nice to have: - Knowledge of web-based frameworks (Flask, Django, FastAPI) - Knowledge of and experience in working with Kubernetes - Experience in working with cloud automation and IaC provisioning tools (Terraform, CloudFormation, etc.) - Experience with Data Engineering / ETL Pipelines (Apache Airflow, Pandas, PySpark, Hadoop, etc.) - Good understanding of application architecture principles We offer: - Competitive compensation - Remote work - Flexible working hours - A team with an excellent...

We are an expanding IT company seeking skilled and experienced data engineering professionals to support ou...years of experience in a data engineering role. Desired (but not required) Skills: - Experience with other data processing technologies such as Apache Flink, Apache Beam, or Apache Nifi. - Knowledge of containerization technologies like Docker and Kubernetes. - Familiarity with data visualization tools such as Tableau, Power BI, or Looker. - Understanding of Big Data tools and technologies like Hadoop, MapReduce, etc. If you possess the necessary skills and experience, we invite you to reach out to us with your CV and relevant information. We are excited to collaborate with you and contribute to the continued success and innovation of our IT company in the field of data en...

I am in immediate need of a full-time Java Spark developer for my project. T...developer for my project. The main goal of the project is data analysis, and I require a developer with mid-level experience. The estimated duration of the project is 6+ months, so I need someone who can commit to a long-term engagement. Ideal skills and experience for this project include: • Expertise in Java Spark • Strong background in data analysis • Experience with big data technologies such as Hadoop and Kafka • Knowledge of distributed systems and cloud computing • Ability to work independently and as part of a team • Strong problem-solving and communication skills If you are a mid-level Java Spark developer looking for a long-term project, please apply with your res...

We are seeking a skilled Big Data Engineer to join our team. The ideal candidate will have experience in data analysis, machine learning, modelling, and pipeline warehousing. Knowledge of Hadoop, Spark, Python, Scala, Java, Storm, Kafka, Flint, Kubernetes, and Docker are essential for this project. The expected duration of the project is more than 3 months. If you have a passion for Big Data and are excited about working in a dynamic team environment, we encourage you to apply.

Quantori is a new company with a long history. We have over twenty years' experience in developing software for the pharmaceutical industry and driving advanced strategies in the world of Big Data revolution. ...- Good written and spoken English skills (upper-intermediate or higher) Nice to have: - Knowledge of web-based frameworks (Flask, Django, FastAPI) - Knowledge of and experience in working with Kubernetes - Experience in working with cloud automation and IaC provisioning tools (Terraform, CloudFormation, etc.) - Experience with Data Engineering / ETL Pipelines (Apache Airflow, Pandas, PySpark, Hadoop, etc.) - Good understanding of application architecture principles We offer: - Competitive compensation - Remote work - Flexible working hours - A team with an excellent...

EMR ssh tunneling ssh -i <path-to-your-key-pair> -L 8020:localhost:8020 -L 8088:localhost:8088 -L 50070:localhost:50070 -L 10000:localhost:10000 -L 50030:localhost:50030 -L 19888:localhost:19888 -L 8080:localhost:8080 -L 4040:localhost:4040 -ND 8157 hadoop@<master-node-public-dns>

...5-6 years of experience in a data engineering role Desired (but not required) Skills: - Experience with other data processing technologies such as Apache Flink, Apache Beam, or Apache Nifi - Knowledge of containerization technologies like Docker and Kubernetes - Familiarity with data visualization tools such as Tableau, Power BI, or Looker - Understanding of Big Data Tools and technologies like Hadoop, Map Reduce etc If you have the skills and experience we're looking for, we'd love to hear from you! Please reach out to us with your CV and experience information. We are looking forward to working with you to help our IT company continue to thrive and innovate in the world of data engineering. ...

...corresponding files to HDFS - Use google trends to get popularity score per each news category - Using the popularity score to simulate read access for each article on HDFS for generating log files Now we have for example a football article with a popularity score/ read access x, based on this score (how many times it got accessed) we categorize each article as HOT, WARM, COLD. With the default hadoop HDFS replica policy 3x, I need to measure system performance and storage. Then I need to modify existing files of the system to be like this HOT files get replicated 3x, WARM 2x and COLD 1x and if a new file is inserted to the system it should be added as HOT 3x. So we measure the performance and storage here. Then I need a machine learning model trained on the articles dataset to b...

I need a freelancer who can help me enhance our migration techniques from hive data to Neo4j database. Our hive data size is less than 100 GB and we are currently using Hadoop for migration. However, we are facing speed and performance issues with the current migration process. Ideal Skills and Experience: - Strong experience in migration techniques from hive data to Neo4j database - Expertise in Hadoop and other tools or frameworks for migration - Knowledge of optimizing speed and performance in migration process - Familiarity with data consistency and complexity of the process The project requires quick and efficient migration techniques to ensure that our data is migrated smoothly and without any loss. Please apply only if you have the ideal skills and experience to...

Hi We need Freelancer who had experience in " Cornerstone, Mysql,Python and Hadoop " It's a Job Supporting Project you need to connect with our consultant through Zoom Meeting and help him to complete the tasks you need to work on his system(Remotely) by taking mouse controls using Zoom 2hrs/day 5days/week For that we will pay you USD 100$ Month Timing - anytime before 10a.m IST Or anytime after 7p.m IST will be fine

We have the Requirements of Hadoop Trainer for 10 days in roorkee location (Uttrakahnd) . Total participants will be 37 . We will provide the Accommodation & Food to the Trainer & also Travel from their origin point to Roorkee & Back.

...In-depth knowledge of Apache Spark and its various components, including Spark Core, Spark SQL, and Spark Streaming. ✅ Familiarity with data warehousing concepts and technologies, such as dimensional modeling and OLAP. ✅ Expertise in ETL (Extract, Transform, Load) processes, data pipelines, and data integration tools. ✅ Understanding of distributed systems and experience with technologies like Hadoop and HDFS. ✅ Proficiency in programming languages commonly used in data engineering, such as Python or Scala. ✅ Ability to design and optimize data architectures, ensuring scalability, performance, and reliability. ✅ Experience with cloud platforms like AWS, GCP, or Azure, and their respective data engineering services. ✅ Familiarity with SQL and NoSQL databases, and proficiency in wr...

We need to deliver 10 days training on Hadoop to engineering Students. So we need a person who can deliver this training on physical Mode in Roorkee City. Accommodation & Food and travel we will take care

i am are looking for a data engineer to assist us with our data processing. The ideal candidate will have experience working with Spark Scala and strong java coding will be able to assist me with my data processing tasks. The ideal candidate will have experience with hadoop ,Spark Scala and strong Java and in putty environment will be able to assist me with my data processing tasks.

...and Business Intelligence (BI). Deep understanding over relational database systems. Expertise on efficient data flux implementations over distributed frameworks. Native level or proficient in English (Written and spoken). Is a must: Engineering background (Computer Science, Electrical and Electronics, Physics, Control, etc), Mathematics or Statistics. Knowledge on Big Data environments as Hive, Hadoop or Impala. Expertise on centralized data repositories (Data Warehouse, Data Lake, Data Mart) by GCP, Azure, AWS or Snowflake. Programming expertise in Python or R. Expertise over other dashboard tools (Power BI, Looker, Dash). Expertise on software versioning tools (Git). Expertise with agile methodologies. Project objective: The client has a web application: the “Vendor Por...

Hello, Task-1: Create code of Apriori Algorithm and implement in Apache hive. As per my side hadoop and hive already installed just you need to implement only. Thanks

Hello, migrating Hadoop workflows to Aws airflow. In on prem workflows there are some shell scripts . Need help to migrate those to airflow using python.

AI-Powered Auto Marketplace: A marketplace for buying and selling cars powered by advanced AI technology. Our marketplace wil...behavior. Our goal is to make the car buying and selling process easier, faster, and more transparent. Tools & Technoligies: • Machine Learning Frameworks: TensorFlow, Scikit-Learn, Caffe, Theano • Natural Language Processing Libraries: NLTK, CoreNLP, OpenNLP • Deep Learning Libraries: Keras, PyTorch, MXNet • Programming Languages: Python, R, Java, C++ • Data Storage: MySQL, MongoDB, Cassandra, Hadoop • Cloud Computing Platforms: AWS, Google Cloud Platform, Microsoft Azure • Visualization Tools: Tableau, D3.js, Matplotlib, Seaborn • DevOps Tools: Docker, Ansible, Jenkins, Kubernetes This Phase includes the arch...

I need to complete my project. already most of task completed just one step is away. Right now first we need to resolve why insert operation is not working then we will go further.

...outside of Hadoop to generate these results. Re: the “plot” required of the second program: one way to do this is to generate the necessary info (cloud-side) and then cut-and-paste it into Excel running on your laptop (and then make the visual rep of this). a. Two receive the full credit, the document should contain screenshot of Hadoop executions! 2. A zip file containing all of your Map/Reduce programs Now, using Python, write the two separate Map/Reduce programs (identified earlier) using Hadoop 2.10.1 on GCP to compute the following using the sample HVAC data You are required to run your program(s) via Hadoop 2.10.1 pseudo-distributed mode on an GCP instance. You cannot use any “add-on” to Hadoop (such as Hive). The great ...

...need help deciding. Additionally, the estimated size of the data warehouse is not yet determined, as I am still evaluating the requirements. The ideal candidate will have experience with warehouse architecture and management, as well as possess a thorough understanding of SQL and other database development techniques. Specific experience with a data warehouse environment and languages, such as Hadoop, Pig, etc. could also be beneficial. The successful freelancer should be able to develop a secure, reliable, and efficient data warehouse capable of meeting our business demands. Furthermore, experience in data analysis, project management, and other related skills will also be appreciated. If this project sounds like it's right up your alley, and you have the skills and exp...

Im having a Technology Training company, i need trainers to help me complete the class based on the instruction provided. Categories : 1. Data analyst - Excel and PowerBI 2. Data engineer - Database / ETL / Data mart building 3. Data scientist - Python / Prediction / optimization 4. Cloud Data Expert - Aws / Databricks / Hadoop So if you are housewife / trainer interested to take part time training in any of the below areas send me you resume with you monthly training quote - expected 6 hrs. per week of training.

Looking for Big data engineer with expert level experience in python, pyspark, sql, Hadoop, airflow and aws services like EMR, s3

Using Spark , Hadoop and Bash to manage data and solve different tasks 1. (Tasks from Section 2) 2. (Tasks from Section 3.1) 3. (Tasks from Section 3.2) 4. (tasks from Section 4.1) 5. (tasks from Section 4.2) • Data Collection (Bash) • Data Managenent (MySQL/MongoDB) • Data Processing using Hadoop • Data Processing using Spark

a continuation to Project I, perform predictive analytics based on the GTD and produce relevant insights (minimum 5 key findings). Each key finding should be supported by relevant visualizations. Additional data sources may be used (or provided by instructor) for this particular steps. –...predictive analytics based on the GTD and produce relevant insights (minimum 5 key findings). Each key finding should be supported by relevant visualizations. Additional data sources may be used (or provided by instructor) for this particular steps. – CLO 4 Critique Assignment. Read and perform critical analysis on the following paper: Analyzing Relationships in Terrorism Big Data Using Hadoop and Statistics by Strang & Zhaohao (2017) – CLO 5

Hi, We are a training institute a startup. We would like prepare a self learning module, where a students can access it and can learn it by himself / herself. We have a Learning Management System where the course module can be installed. We want a training module which consist of Hadoop Basic Training and we can also provide study material for reference.

5+Years of experience working as a Data Engineer Primary Skills - PySpark, AWS, EMR ,hadoop,SQL · Strong experience in developing data processing task using Spark on cloud native services like Glue/EMR · Strong Data and Big Data skills with experience working on Data projects · Strong data warehousing skills is mandatory. · Strong experience in designing, developing Data solutions both on premise and on cloud · Strong knowledge on optimizing workloads developed using Spark/SparkSQL · Experience in EMR, Hadoop, and AWS services and Pyspark · Proficiency with Data Processing: HDFS, Hive, Spark, Python. · Strong analytic skills related to working with structured, semi ...

Please Sign Up or Login to see details.

No Of Tables = 1 Table name = t_shopping_electronics No Of Columns = 6 This is similar to online/in-person shopping/checkout experience. customer can buy multiple things in one transaction. --- Transaction_id, Name, Product, Cost, Brand, Date --- Inputs Transaction ID is same for the trip/receipt Customer can buy multiple products in the same shopping trip . 1 receipt = 1 transaction ID Requirement Extract data where Customer buys TV + HOME THEATER and DOES NOT buy Watch group by Date, Brand WATCH can be treated as ERROR ( but we need to start with TV + HOME THEATER - because a lot of business data is embedded in these 2 rows ) --- If Customer buys only TV + HOME THEATER- INCLUDE If Customer buys only TV + HOME THEATER and other non-watch products - INCLUDE If Customer buys o...

Position: Hadoop Big Data Developer Type: Remote Screen Sharing Duration: Part-Time Monday to Friday 4 hours a day Salary: $700 Per Month (57,000 INR per month) Start Date: ASAP We are looking for a Hadoop Big Data Developer with experience in Hadoop, Spark, Sqoop, Python, Pyspark, Scala, Shell Scripting, and Linux. We are looking for someone who can work in the EST time zone connecting via remote i.e zoom, google meet on a daily basis to assist in completing the tasks. Here we will be working via screen share remotely, no environment setup will be shared.

Require help for a collage project which requires creating four nodes in a single system and upload a data set. Perform some basic queries to retrieve info from HDFS

A project that recommends movies based on collaborative, content and hybrid based filtering. Must use hadoop

We are looking for Big data engineer trainer who has real time experience in Python, SQL, Pyspark, Hadoop concepts and good knowledge on AWS services like Glue, Athena, Lambda, EMR, S3, Apache airflow

For a assignment of big data hadoop using python

Want someone who can make project of big data python hadoop

This is strictly a WFO job. Only local candidates from Chennai OR those who are ready to relocate to Chennai should apply. Duration: 6 months plus Role1: Bigdata, Hadoop,sprk,airflow, CICD, python (scripting), devops. 3-8 years experience. Role 2: Data product manager - Tableau, SQL queries with managerial skills 5-8 years experience. Role 3: BI engineer - SQL,SQL Server, ETL, Tableau, data modelling, scripting, agile, python 5-8 years experience Role 4: Data Engineer - Big data, Hive, Spark, Python 3-7 years experience Very good communication skills is mandatory Must be ready to work from our office in Chennai Timings: 9 hours, IST business hours, Monday - Friday.

...задачи по ETL 50%, а также 10% ML и 40% DS. Стек: SQL+PL/SQL Greenplum, Teradata, MSSQL, MySQL, SQLite,… DWH+ETL работа с хранилищами данных Hadoop Hive, Impala, Spark, Oozie, … Python pandas, numpy, pyspark, … Machine Learning Что делать: Рефакторинг прототипов моделей машинного обучения от команды DataScience – адаптация кода к пайплайну поставки моделей в промышленную эксплуатацию с сохранением результатов и оценки моделей в хранилище Greenplum (MLOps) Проектирование и разработка корпоративной аналитической платформы Разработка процессов построения пакетной и near real time аналитики Разработка, поддержка и оптимизация ETL на платформах Greenplum и Hadoop Поддержание технической документации в актуальном состоянии

wordpress site build + customize So php, node.js, Java, .NET Hadoop?

/ fs needs reconfiguration for hdfs layout

Includes Java in coding part and other than that we require experience in Aws, Hadoop, and Spark

Hi, I am looking data analyst job timing is US healthcare claims and provide required support in Excel,sql,Db2,Hadoop,Informatica (basics).Daily one or two hours

We are is searching for an accountable, multitalented data engineer to facilitate the operations of our data scientists. The data engineer will be responsible for employing ...technological advancements that will improve the quality of your outputs. Data Engineer Requirements: Bachelor's degree in data engineering, big data analytics, computer engineering, or related field. Master's degree in a relevant field is advantageous. Proven experience as a data engineer, software developer, or similar. Expert proficiency in Python, C++, Java, R, and SQL. Familiarity with Hadoop or suitable equivalent. Excellent analytical and problem-solving skills. A knack for independence and group work. Scrupulous approach to duties. Capacity to successfully manage a pipeline of duties with ...