Resident Solution Architect, Databricks

📋 Description

• Design and oversee the implementation of data platforms based on Databricks, utilizing best practices in Lakehouse and Medallion architecture.

• Offer hands-on technical leadership across data engineering, analytics, and AI-driven use cases within the Databricks environment.

• Establish and uphold standards for data governance, security, and access control through Databricks Unity Catalog and enterprise data management principles.

• Facilitate and support AI/ML workloads, incorporating ML pipelines and model lifecycle integration into the Databricks ecosystem.

• Enhance platform performance, scalability, and cost efficiency with a focus on FinOps.

• Collaborate effectively with business leaders, architects, and engineering teams to ensure platform capabilities meet enterprise data and analytics requirements.

• Act as a resident technical expert, providing architectural guidance, mentoring teams, and guaranteeing solution quality and consistency.

⛳️ Requirements

• Over 10 years of comprehensive experience in data engineering, data architecture, or analytics engineering roles.

• More than 5 years of hands-on experience with Databricks, including Delta Lake, Unity Catalog, Workflows/Jobs, and Lakehouse architecture.

• Databricks Professional Certification is mandatory.

• Strong knowledge of Apache Spark (preferably PySpark), Python, and advanced SQL.

• Demonstrated experience in designing and implementing production-grade, large-scale data pipelines.

• Practical experience with at least one major cloud platform (Azure preferred; AWS or GCP acceptable).

• Familiarity with data modeling, Medallion architecture, and designing enterprise data platforms.

• Experience with CI/CD pipelines, infrastructure automation tools (such as Terraform), and Agile delivery methodologies.

• Excellent communication and stakeholder engagement skills, with a background in regulated enterprise environments.

🏝️ Benefits

• Health insurance

• Retirement plans

• Paid time off

• Flexible work arrangements

• Professional development