Data Engineer

Vakansiyanın detalları

Design, develop, and optimize ETL/ELT workflows using ETL/ELT tools (Apache Spark, DBT, Apache Airflow, Apache Nifi) from various sources to big data platforms.
Design, develop, and optimize ETL/ELT workflows using SAP SLT, SAP BW for data replication from SAP ERP systems to SAP HANA and big data platforms.
Develop data models for big data storage and processing.
Develop dimensional data models for data warehouse
Develop data marts as a data products for data science and business intelligence workloads
Write and optimize code in programming languages such as Python, Go, Java, or Scala to process large datasets.
Automate data processing tasks and develop custom data solutions.
Develop streaming ELT/ETL data pipelines using Apache Kafka and Apache Spark
Develop and finetune existing CI/CD pipelines using Gitlab
Develop and finetune process orchestration using Apache Airflow
Implement data quality checks and validation procedures.
Ensure compliance with data governance policies, including data security and privacy standards.
Work with cross-functional teams, including data scientists, analysts, and business stakeholders, to meet data requirements.
Provide technical support and troubleshoot data-related issues.

Bachelor's degree in Computer Science, Information Systems, or related field; Master's degree is a plus.
3-5 years of experience in data engineering.
Experience with SAP analytical stack (SAP BW, SAP HANA, SAP BO)
Experience with big data technologies (e.g., Hadoop, Spark, Lakehouse).
Basic experience with data streaming technologies (Kafka and Spark Streaming).
Strong programming skills in Python, Go, Java, or Scala.
Experience with Apache Airflow.
Solid understanding of ETL/ELT processes and data warehousing concepts
Solid understanding of SQL and NoSQL databases (Oracle, MS SQL, PostgreSQL, SAP HANA, Cassandra)
Experience with containerization (Docker, Kubernetes as developer)
Basic understanding of oil and gas industry processes and data requirements is a plus
Excellent problem-solving skills and attention to detail.
Strong communication and teamwork abilities.