Middle Data Engineer – Azure Databricks

This is a fully remote position, open to applicants in Poland.

📋 Description

• Design, develop, and maintain data pipelines utilizing Azure Databricks.

• Create and enhance data transformations with PySpark and SQL within Databricks.

• Implement and uphold Lakehouse architectures using Delta Lake.

• Develop ETL/ELT pipelines that are orchestrated through Azure Data Factory.

• Integrate data from various sources into the data platform and analytical layers.

• Design and sustain data models and data warehouse structures for analytics purposes.

• Ensure the quality, scalability, and performance of extensive data processing pipelines.

• Collaborate with BI teams to facilitate Power BI and reporting platforms.

• Support and enhance existing SQL Server data platforms and ETL solutions (SSIS) as necessary.

• Contribute to the architecture design of modern cloud-based data systems.

⛳️ Requirements

• Over 3 years of experience in Data Engineering or Data Warehouse development.

• Proficiency with Azure Databricks.

• Experience in developing data pipelines using PySpark and Spark SQL.

• Strong understanding of distributed data processing and big data concepts.

• Familiarity with Delta Lake and Lakehouse architectures.

• Advanced SQL skills and experience with SQL Server relational databases.

• Experience in constructing data pipelines using Azure Data Factory.

• Capability to manage large datasets and optimize performance.

• Nice to have:

• Experience with Spark optimization strategies (partitioning, caching, cluster tuning).

• Familiarity with structured streaming in Databricks.

• Knowledge of CI/CD pipelines for data platforms (Azure DevOps).

• Understanding of Power BI.

• Experience in transitioning from traditional ETL processes to cloud-based architectures.

🏝️ Benefits

• Culture of Relentless Performance: join an unstoppable technology development team with a 99% project success rate and more than 30% year-over-year revenue growth.

• Competitive Pay and Benefits: enjoy a comprehensive compensation and benefits package, including health insurance and a relocation program.

• Work From Anywhere Culture: take advantage of the flexibility that comes with remote work.

• Growth Mindset: benefit from a variety of professional development opportunities, including certification programs, mentorship, talent investment programs, internal mobility, and internship opportunities.

• Global Impact: collaborate on significant projects for top global clients and help shape the future of industries.

• Welcoming Multicultural Environment: be part of a dynamic, global team and thrive in an inclusive and supportive work atmosphere with open communication and regular team-building social events.

• Social Sustainability Values: join our sustainable business practices focused on five pillars, including IT education, community empowerment, fair operating practices, environmental sustainability, and gender equality.