Middle Data Engineer, Azure Databricks

This is a fully remote position, open to applicants in Spain.

📋 Description

• Design, create, and manage data pipelines utilizing Azure Databricks.

• Construct and enhance data transformations with PySpark and SQL within Databricks.

• Implement and oversee Lakehouse architectures leveraging Delta Lake.

• Develop ETL/ELT pipelines coordinated through Azure Data Factory.

• Integrate data from various sources into the data platform and analytical layers.

• Design and manage data models and data warehouse frameworks for analytics.

• Ensure data quality, scalability, and the performance of large-scale data processing pipelines.

• Collaborate with BI teams to support Power BI and reporting platforms.

• Provide support and enhance existing SQL Server data platforms and ETL solutions (SSIS) as needed.

• Contribute to the design of advanced cloud-based data architectures.

⛳️ Requirements

• Over 3 years of experience in Data Engineering or Data Warehouse development.

• Proficiency with Azure Databricks.

• Experience in developing data pipelines using PySpark and Spark SQL.

• Strong grasp of distributed data processing and big data principles.

• Experience with Delta Lake and Lakehouse architectures.

• Robust SQL expertise along with experience in SQL Server relational databases.

• Experience in constructing data pipelines via Azure Data Factory.

• Capability in managing large datasets and optimizing performance.

• Preferred: Experience with Spark optimization techniques such as partitioning, caching, and cluster tuning.

• Preferred: Experience with structured streaming in Databricks.

• Preferred: Knowledge of CI/CD pipelines for data platforms (Azure DevOps).

• Preferred: Familiarity with Power BI.

• Preferred: Experience in migrating from traditional ETL processes to cloud architectures.

🏝️ Benefits

• Culture of Relentless Performance: join an unstoppable technology development team with a 99% project success rate and more than 30% year-over-year revenue growth.

• Competitive Pay and Benefits: enjoy a comprehensive compensation and benefits package, including health insurance and a relocation program.

• Work From Anywhere Culture: make the most of the flexibility that comes with remote work.

• Growth Mindset: reap the benefits of a range of professional development opportunities, including certification programs, mentorship, talent investment programs, internal mobility, and internship opportunities.

• Global Impact: collaborate on impactful projects for top global clients and shape the future of industries.

• Welcoming Multicultural Environment: be a part of a dynamic global team and thrive in an inclusive and supportive work environment with open communication and regular team-building social events.

• Social Sustainability Values: join our sustainable business practices focused on five pillars, including IT education, community empowerment, fair operating practices, environmental sustainability, and gender equality.