Senior Databricks Engineer

📋 Description

• Design, develop, and sustain intricate ETL pipelines within Databricks, ensuring scalable and high-performance data integration across various source systems.

• Implement and enhance medallion architecture in Databricks by creating structured data zones (raw, curated, and trusted) to facilitate governed, enterprise-level reporting.

• Create and improve dimensional data models that deliver analytics-ready business perspectives and support automated dashboards and KPI reporting frameworks.

• Work collaboratively with cross-functional teams, including data stewards, IT departments, and business stakeholders, to convert operational requirements into effective technical solutions while proactively identifying dependencies and fostering alignment.

• Contribute to architectural and technical decisions by suggesting best practices, questioning assumptions when necessary, and ensuring the scalability, durability, and flexibility of the data platform.

• Actively identify and resolve integration challenges, data quality issues, and process bottlenecks, while providing actionable insights and constructively highlighting potential project risks or inefficiencies.

• Assist in documentation and knowledge-sharing efforts to empower teams and clients to independently maintain and enhance data solutions over time.

⛳️ Requirements

• 5+ years of practical experience in Databricks development, supported by relevant certifications (proof required).

• Established expertise in constructing, optimizing, and troubleshooting production-grade ETL pipelines.

• Strong background in designing and implementing medallion architecture models (raw, curated, and trusted layers) in Databricks environments.

• Comprehensive understanding of dimensional data modeling to support business intelligence, enterprise reporting, and single source of truth initiatives.

• Advanced skills in orchestrating data ingestion, transformation, and integration workflows across multiple systems and data formats.

• Ability to interpret operational requirements, translate them into technical solutions, and clearly communicate the business implications of engineering decisions.

• Proven capability to take ownership, work independently, and successfully drive projects in dynamic or ambiguous settings.

• Experience with Azure Data Lake, Azure Data Factory, and associated Azure services is considered a significant advantage.

• Excellent communication and collaboration abilities, with the confidence to offer constructive feedback, challenge assumptions, and advocate for best practices.

• Capacity to adapt swiftly, remain results-oriented, collaborate effectively, maintain a positive demeanor, and lead with empathy.

• Actively contribute to a culture of collaboration, continuous improvement, knowledge sharing, and openness to innovation.

• Demonstrated ability to provide meaningful insights and respectfully challenge when necessary, always focusing on achieving optimal outcomes for the team and project.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Opportunities for professional development and continuous learning.

• Flexible working hours and remote work options.

• A supportive and inclusive company culture that values diversity.