Databricks Data Engineer / Developer

This is a fully remote position, open to applicants in Canada.

📋 Description

• Design, develop, and maintain data pipelines within the Enterprise Data Platform (EDP).

• Construct and enhance ETL/ELT processes for data ingestion, transformation, and integration.

• Employ an Agentic approach to development, ensuring outputs adhere to development standards.

• Create scalable and reusable data solutions utilizing Databricks and cloud-based data technologies.

• Assist in the migration and modernization efforts from HANA environments to a contemporary data platform.

• Collaborate with Data Architects to design scalable data models and platform solutions.

• Develop data transformation logic and workflows to meet business and analytical needs.

• Ensure the quality, integrity, consistency, and performance of data.

• Diagnose data-related issues, bottlenecks, and performance challenges.

• Engage in code reviews, testing, deployment, and release activities.

• Partner with business and analytics teams to comprehend requirements and deliver tailored solutions.

• Contribute to best practices for the platform, documentation, and continuous improvement initiatives.

⛳️ Requirements

• A minimum of 3+ years of practical Databricks development experience.

• At least 4+ years of comprehensive experience in Data Engineering, ETL, or Data Integration.

• Extensive experience in constructing ETL/ELT data pipelines and transformation processes.

• Demonstrated experience working with large and intricate datasets.

• Familiarity with developing data solutions on cloud-based platforms.

• Strong understanding of data warehousing, data lake, and lakehouse concepts.

• Experience in Agile delivery environments.

• Excellent problem-solving and analytical capabilities.

• Outstanding communication and collaboration skills.

• Proficiency with Databricks, ETL/ELT development, data engineering, data integration, SQL, Python, data pipeline development, data transformation, and data warehousing concepts.

• Experience with Agentic frameworks and AI-assisted coding is advantageous.

• Preferred: experience in supporting large data platform initiatives, significant data migrations, Azure Data Services, Databricks Lakehouse architecture, modern data engineering frameworks, AI/GenAI/Machine Learning/advanced analytics, enterprise or consulting environments, as well as data governance, metadata management, and data quality frameworks.

• Preferred technical knowledge includes Azure Data Factory, Delta Lake, Spark/PySpark, Azure Synapse, data lake architecture, cloud data platforms, data governance and data quality, and CI/CD for data solutions.

• Applicants must specify their work authorization status in Canada.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Opportunities for professional development and continuous learning.

• Flexible working hours and remote work options.

• Comprehensive health and wellness benefits.

• Collaborative and inclusive work environment.