Senior Data Engineer, Product Data Systems

This is a fully remote position, open to applicants in Canada.

📋 Description

• Design, develop, test, deploy, and manage production services for data ingestion, normalization, enrichment, identity resolution, scoring, and intelligence delivery.

• Construct maintainable pipeline workers, event consumers, APIs, scheduled processes, and supporting libraries.

• Take ownership of the entire software lifecycle, encompassing architecture, implementation, testing, deployment, monitoring, incident response, and continuous improvement.

• Establish reusable engineering patterns for the team.

• Create asynchronous workflows with well-defined data, event, and work contracts.

• Design systems to handle duplicate delivery, ordering constraints, idempotency, retries, timeouts, partial failures, dead-letter handling, backpressure, and recovery.

• Ensure that pipeline state and failures are observable, with reconciliation processes for missing, delayed, duplicated, or inconsistent processing.

• Facilitate safe replay and reprocessing of data.

• Maintain source evidence, provenance, lineage, processing context, and applicable versions.

• Define validation and quality controls at service interfaces and safely evolve schemas, contracts, and processing logic.

• Ensure tenant isolation while integrating shared intelligence with customer-specific evidence, configurations, and conclusions.

• Implement authorization, retention, deletion, audit, GDPR, and SOC 2 compliance requirements.

• Assess managed services, open-source components, existing capabilities, and specialized services.

• Contribute to architectural development through working software, proposals, prototypes, and technical reviews.

• Collaborate with Product, Platform Engineering, Threat Research, Data Science, Security, and customer-facing teams.

• Translate product requirements into technical contracts, communicate trade-offs, and provide mentorship to engineers.

⛳️ Requirements

• Extensive hands-on experience in building and operating data-intensive software for externally used SaaS products.

• Strong specialization in software engineering focused on data systems.

• Production development experience with Go, Scala, Rust, Java, or another comparable backend-service language.

• Willingness to primarily work in Go; prior experience with Go is strongly preferred.

• Proficiency in Python and SQL.

• Understanding of relational, document, graph, key-value, analytical, and object-storage models along with their trade-offs.

• Knowledge of distributed-systems challenges, including asynchronous processing, delivery semantics, concurrency, backpressure, idempotency, consistency, recovery, and failure isolation.

• Experience in designing or operating event-driven systems using Kafka, Azure Event Hubs, AWS Kinesis, or similar messaging infrastructure.

• Familiarity with containers, cloud infrastructure, automated testing, CI/CD, infrastructure automation, monitoring, and production operations.

• Ability to reason about provenance, replay, data quality, multi-tenant isolation, shared data, private customer contexts, and authorization boundaries.

• Capability to evaluate unfamiliar technologies based on engineering principles, communicate effectively, constructively challenge weak assumptions, and take ownership of delivery outcomes.

• Authorization to work in the country of employment; this position is not eligible for visa sponsorship.

• Valuable experience in cybersecurity, threat intelligence, fraud and abuse prevention, evidence-intensive domains, data provenance, explainability, auditability, regulated data systems, graph processing, Neo4j, Azure Data Explorer, Azure Data Lake Storage, machine learning pipelines, legacy workload migration, or customer-facing data systems.

🏝️ Benefits

• Flexible time off.

• Sick days.

• All public holidays.

• Stock options.

• Benefits tailored to the country of employment.

• Reasonable accommodations for qualified individuals with disabilities as needed.