Senior Data Engineer

This is a fully remote position, open to applicants in Canada.

📋 Description

• Design and execute comprehensive data architectures, encompassing data lakes, data warehouses, and analytics platforms.

• Establish data integration and transformation strategies that prioritize scalability, security, and performance.

• Convert business needs into technical solutions that support analytics, reporting, and AI initiatives.

• Create data models, ETL/ELT pipelines, and frameworks for both structured and unstructured data.

• Offer technical leadership and mentorship to data engineers and developers.

• Advocate for best practices in data management and governance.

• Ensure adherence to data governance, security, and privacy standards across various platforms.

• Enhance existing data architectures and processes for improved performance and reliability.

• Keep up to date with cloud data services and emerging technologies.

• Serve as a trusted advisor to clients regarding architectural decisions and data modernization best practices.

• Collaborate with business stakeholders, data engineers, and analytics teams to ensure solutions align with client objectives.

⛳️ Requirements

• Over 5 years of experience in data architecture, data engineering, or analytics solution design.

• Practical experience with data lake and warehouse technologies, including Databricks, Snowflake, Redshift, and Synapse.

• Profound understanding of data modeling, data integration, and ETL/ELT design.

• Proficient in SQL and familiar with one or more programming languages, such as PySpark/Python or Scala.

• Experience in complex data transformations and optimization within Spark.

• Knowledge of data governance, security, and privacy best practices.

• Skilled in designing, implementing, and optimizing large-scale ingestion pipelines utilizing Databricks Autoloader.

• Practical experience in building and managing reliable, self-managing ETL/ELT pipelines using Delta Live Tables.

• Experience in developing high-throughput, low-latency streaming data ingestion solutions with Apache Kafka, Spark Structured Streaming, and Databricks Streaming.

• Extensive experience in applying and enforcing Medallion architecture within a Databricks environment.

• Experience in designing and implementing CI/CD pipelines with tools such as Azure DevOps, GitHub Actions, or GitLab CI for Databricks workflows, notebooks, and cluster configurations.

• Experience in planning and executing data migration projects from traditional data warehouses to the Databricks Lakehouse.

• Strong working knowledge of at least one major cloud provider, AWS or Azure, including data storage, networking, and security concepts relevant to Databricks deployment.

• Ability to engage with clients, present technical solutions, and convey complex ideas effectively.

• Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field.

• Excellent problem-solving, communication, and collaboration skills.

• Nice to have: AI/ML integration and data science workflows.

• Nice to have: experience with data cataloging and metadata management tools.

• Nice to have: previous consulting or client-facing experience in a technology services firm.

🏝️ Benefits

• Supportive, collaborative team environment.

• Great Place to Work recognition.

• Opportunities to deepen expertise and enhance skills.

• Meaningful work addressing real-world business challenges.

• Professional development through exposure to modern platforms, cloud services, AI, and automation consulting.

• Full-time employment.