I am Helin. I have more than 4 years of experience in a large e-commerce company and the TV industry as a data engineer and data scientist. I used state-of-art techs in big data, data analysis, and data mining areas. I also have an academic background, I am continuing my master's education in data science.
->Stream and batch processing Apache Spark, Apache Kafka, and Apache Flink in Scala and Pyspark
-> Building API s in Golang and Python
-> Use NewRelic, Kubernetes, and Grafana service
-> Create job schedules using Apache Airflow DAGs, Gitlab CI/CD pipelines and AWS Glue schedules
->ETL and ELT operations data from a variety of sources (S3, Redshift, RDS) with pyspark jobs(AWS Glue, AWS EMR)
-> Optimize time and memory usage of Spark Applications (control DPU and execution time)
-> Audio fingerprint matching on big data using pyspark
-> Video matching(extracted features with deep learning models) on big data using pyspark(scale large join operations with approximate similarity join using LSH features )