Senior Data Engineer

This is a fully remote position, open to applicants in Poland.

📋 Description

• Design and implement scalable, reliable, and reusable data pipelines utilizing Azure Data Factory and Apache Airflow

• Develop data transformations in Azure Databricks leveraging PySpark and SQL

• Create and enhance data products adhering to Bronze, Silver, and Gold Medallion Architecture layers

• Optimize the performance, cost, and reliability of data workloads

• Identify critical questions during business maturation to mitigate risks prior to development

• Engage in architecture reviews and contribute to technical decision-making

• Advocate for engineering excellence through code reviews and design assessments

• Collaborate with Architects and Product Owners to facilitate the technical maturation of data products

• Provide support to the Run/Operations team for production pipelines and data products

• Diagnose and resolve data incidents, failures, and performance challenges

• Take part in incident management, root cause analysis, and post-mortem evaluations

• Ensure proper handover, documentation, and operational readiness for developed solutions

• Contribute to the enhancement of runbooks, monitoring, and operational processes

⛳️ Requirements

• Over 6 years of experience in data engineering

• Strong technical proficiency in Python (PySpark)

• Proficient SQL skills

• Practical experience with Azure-based Data Platform

• Familiarity with Databricks

• Experience with Unity Catalog

• Knowledge of Apache Airflow

• Proficient in Azure Data Factory

• Experience with reporting and BI tools such as Power BI, Tableau, or similar

• Comprehensive understanding of data modeling fundamentals, key architectures, and design patterns

• Familiarity with Data Warehouse, Data Lake, and Data Lakehouse architectures

• Knowledge of Medallion architecture

• Understanding of metadata-driven architecture

• Familiarity with ETL/ELT design patterns

• Insight into partitioning, optimization, and performance

• Understanding of batch and streaming pipelines

🏝️ Benefits

• Opportunities for professional growth and development

• Competitive salary and performance-based bonuses

• Comprehensive health and wellness benefits

• Flexible work hours and remote work options