Senior Data Engineer

This is a fully remote position, open to applicants in Canada.

📋 Description

• Design and execute comprehensive data architectures, encompassing data lakes, data warehouses, and analytics platforms.

• Establish scalable, secure, and high-performance strategies for data integration and transformation.

• Work alongside business stakeholders, data engineers, and analytics teams to convert requirements into technical solutions.

• Create data models, ETL/ELT pipelines, and frameworks for both structured and unstructured data.

• Offer technical guidance and mentorship to data engineers and developers.

• Ensure adherence to data governance, security, and privacy regulations.

• Enhance existing data architectures and processes to improve performance and reliability.

• Keep updated with cloud data services and emerging technologies.

• Serve as a trusted advisor to clients regarding architectural decisions and best practices in data modernization.

⛳️ Requirements

• Over 5 years of experience in data architecture, data engineering, or analytics solution design.

• Practical experience with data lake and warehouse technologies, including Databricks, Snowflake, Redshift, and Synapse.

• Profound knowledge of data modeling, data integration, and ETL/ELT design.

• Expertise in SQL and proficiency in at least one programming language: PySpark/Python or Scala.

• Experience with complex data transformations and optimization using Spark.

• Strong understanding of data governance, security, and privacy best practices.

• Demonstrated experience in designing, implementing, and optimizing large-scale ingestion pipelines with Databricks Autoloader.

• Practical knowledge of building and managing reliable, self-managing ETL/ELT pipelines utilizing Delta Live Tables.

• Experience in creating high-throughput, low-latency streaming data ingestion solutions using Apache Kafka, Spark Structured Streaming, and Databricks Streaming.

• Extensive experience applying and enforcing the Medallion architecture (Bronze, Silver, Gold layers) within Databricks.

• Experience in designing and implementing CI/CD pipelines using tools like Azure DevOps, GitHub Actions, or GitLab CI for Databricks workflows, notebooks, and cluster configurations.

• Skilled in planning and executing data migration projects from traditional data warehouses to the Databricks Lakehouse.

• Strong familiarity with at least one major cloud provider, AWS or Azure, including data storage, networking, and security concepts pertinent to Databricks deployment.

• Ability to engage with clients, articulate technical solutions, and convey complex ideas clearly.

• Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related discipline.

• Exceptional problem-solving, communication, and collaboration skills.

🏝️ Benefits

• Supportive, driven, and collaborative team environment.

• Great Place to Work recognition.

• Opportunities to enhance skills and deepen expertise.

• Meaningful work addressing real-world business challenges.

• Career exposure to AI and automation consulting, cloud, data, and business application technologies.

• Full-time employment.