Senior Data Engineer

📋 Description

• Design, construct, and manage scalable batch and streaming data pipelines.

• Lead architectural choices for data platforms based on Lakehouse principles.

• Develop and orchestrate workflows utilizing Apache Airflow.

• Create transformations and datasets ready for analytics using DBT.

• Develop and sustain real-time data pipelines using Kafka.

• Utilize Databricks for extensive data processing and advanced analytical tasks.

• Design and enhance storage solutions using Apache Iceberg and Lakehouse architecture.

• Ingest and manage data from various sources with tools like Fivetran for a managed data lake.

• Construct and uphold a semantic layer for reliable reporting and self-service analytics.

• Implement data quality frameworks, observability, and automated testing processes.

• Optimize performance, scalability, and costs across AWS services (Athena, EC2, etc.).

• Collaborate with BI, product, and engineering teams to provide actionable data solutions.

• Mentor junior engineers and contribute to best practices and engineering standards.

• Propel enhancements in developer productivity and pipeline reliability.

⛳️ Requirements

• Over 7 years of experience in Data Engineering or a related discipline.

• Strong expertise in Python and SQL.

• Extensive experience with Apache Airflow and workflow orchestration.

• Proficient in DBT for data transformation and modeling.

• Solid hands-on experience with Databricks.

• Significant experience in building streaming pipelines (Kafka or similar technologies).

• Practical experience with data ingestion tools such as Fivetran.

• Experience in developing automated QA, monitoring, and observability for data lakes / Lakehouses.

• Strong understanding of Lakehouse architecture and Apache Iceberg.

• Experience in implementing data quality, testing, and observability frameworks.

• Familiarity with the AWS ecosystem (Athena, EC2, S3, etc.).

• A robust foundation in data modeling and semantic layer design.

• Proven track record in designing scalable systems and influencing technical direction.

🏝️ Benefits

• Flexibility to work in your preferred environment within your country of employment – whether in-office, remote, or hybrid.

• Ongoing investment in your professional growth and development.

• Immediate access to a comprehensive health and wellness benefits package, including an annual wellness stipend.

• 401k plan with up to a 4% match and immediate vesting.

• Flexible and generous time-off policy (FTO).

• Employee Stock Purchase Program.