Databricks Architect

This is a fully remote position, open to applicants in Canada.

📋 Description

• Design and develop scalable solutions based on Databricks for enterprise clients.

• Lead comprehensive implementations of cloud data platforms utilizing AWS and/or Azure.

• Create and construct production pipelines with Databricks Workflows and Spark Declarative Pipelines.

• Develop scalable ETL/ELT pipelines leveraging Spark, PySpark, and Databricks Workflows.

• Collaborate directly with client stakeholders, including C-level executives, to comprehend business requirements and convert them into technical solutions.

• Offer technical leadership and direction to data engineering teams.

• Establish CI/CD pipelines and infrastructure-as-code for Databricks resources.

• Foster collaborative development through Git workflows, pull requests, and code reviews.

• Work alongside analytics, engineering, and business teams.

• Act as the primary technical liaison for clients.

⛳️ Requirements

• Over 6 years of experience in Data Engineering, Cloud Architecture, or similar roles.

• More than 3 years of practical experience with Databricks.

• Strong expertise in PySpark, Spark SQL, Python, and SQL.

• Extensive experience with Delta Lake, Unity Catalog, Delta Live Tables, and Databricks Jobs.

• Demonstrated success in delivering Databricks projects within a consulting or professional services context.

• Solid understanding of data lake concepts and formats, including Delta, Iceberg, and Parquet.

• Experience in designing and managing Unity Catalog for governance, access control, and lineage.

• Familiarity with medallion architecture patterns.

• Practical experience with Git version control, pull requests, code reviews, and collaborative development processes.

• Experience with CI/CD and infrastructure-as-code for Databricks resources, including Terraform, Asset Bundles, and GitHub Actions.

• Experience with MLflow for experiment tracking, model registry, deployment, and observability.

• Knowledge of cloud platforms.

• Experience in client-facing, consulting, or cross-functional environments.

• Proven track record in leading technical projects or mentoring engineering teams.

• Databricks certifications are a plus.

• Familiarity with agentic AI workflows, Agentbricks, LangChain, and Claude agentic workflows is a bonus.

🏝️ Benefits

• Continuous opportunities for exposure and learning.

• Flexible work location that promotes balance between personal life and work.

• Influence and agency in shaping the company’s overall strategy and direction.

• Collaboration with a high-achieving team.

• Competitive base salary.

• Target bonus ranging from 20-25%.

• 401k plan.

• Comprehensive healthcare benefits.

• Paid time off.