Senior Data Engineer

This is a fully remote position, open to applicants in Hungary.

📋 Description

• Design and execute scalable batch and streaming data pipelines utilizing Apache Spark, Kafka, and Flink.

• Construct and sustain Bronze/Silver/Gold medallion architecture within the lakehouse by employing Delta Lake or Iceberg.

• Create and enhance complex SQL and PySpark transformations for extensive datasets.

• Integrate structured, semi-structured, and unstructured data sources into the lakehouse environment.

• Collaborate with data architects to advance physical and logical data models.

• Implement data quality assurance checks and monitoring utilizing Great Expectations or dbt tests.

• Write Infrastructure-as-Code for pipeline environments with Terraform and Helm.

• Engage in code reviews and uphold engineering standards and best practices.

• Diagnose pipeline failures, performance bottlenecks, and data incidents.

• Mentor junior and mid-level data engineers while contributing to internal knowledge sharing.

⛳️ Requirements

• Over 6 years of data engineering experience with a proven record of delivering at an enterprise scale.

• Expertise in Python and SQL.

• Required experience with PySpark.

• Practical experience with Apache Spark, Delta Lake, or Apache Iceberg.

• Familiarity with Apache Airflow, Prefect, or Dagster.

• Strong understanding of AWS Glue, Azure Data Factory, or GCP Dataflow.

• Proficient with Git, CI/CD pipelines, Docker, and Kubernetes.

• Experience with the dbt (data build tool).

• Bachelor's degree in Computer Science, Engineering, or a related technical discipline.

🏝️ Benefits

• Inclusive and accessible work environment.

• Accommodations available upon request for all aspects of the selection process.