Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop, manage, and enhance systems that support both internal analytics and customer-facing analytics platforms.

• Take ownership of the production CDC-fed medallion data lake utilizing Apache Iceberg queried through Trino.

• Operate and improve the Postgres data warehouse, focusing on schema, performance, access controls, and datasets that are ready for analytics.

• Manage CDC ingestion from source databases via message bus, streaming writer, and Iceberg bronze/silver/gold layers.

• Oversee the Trino query layer and table catalog operations.

• Ensure pipeline health by monitoring latency SLAs, schema-drift detection, and source reconciliation.

• Administer GitOps/Terraform infrastructure along with backup and disaster recovery strategies.

• Create consistent schemas, document lineage, and establish clear ownership across the data estate.

• Maintain and enhance Dagster-orchestrated dbt pipelines, which include sensor-triggered and scheduled builds, data-quality tests, and branch-based versioning.

• Manage the BI/reporting layer with per-user, policy-based data access and uniform metric definitions.

• Oversee pipelines that integrate the graph database with the warehouse, lake, and primary databases.

• Transition legacy direct-to-graph services to a shared integration path and refine relational structures into graph-native models.

• Support and expand production genAI workflows, including embeddings, similarity search, and LLM-based extraction and classification.

• Keep the data infrastructure ready for AI applications.

• Collaborate with data science teams and cross-functional analytics stakeholders.

• Engage in incident triage, root-cause analysis, runbook development, and enhancements for reliability.

⛳️ Requirements

• Minimum of 5 years in data engineering, analytics engineering, or data platform engineering.

• Proficient in SQL and relational databases, specifically with Postgres and MongoDB.

• Practical experience with graph databases in a production setting, including integration with data warehouses and lakes.

• Familiarity with open table formats and medallion lake architectures such as Apache Iceberg.

• Experience with distributed SQL engines like Trino or Presto.

• Knowledge of streaming/CDC pipelines such as Kafka or Pulsar, along with tools like Debezium and Flink or similar.

• Strong Python capabilities for pipeline development, automation, and operational tools.

• Experience with dbt orchestrated by modern schedulers such as Dagster or Airflow.

• Proficient in AWS infrastructure including EKS, VPC, IAM, and S3 using infrastructure-as-code solutions like Terraform.

• Experience with CI/CD practices, GitOps tools like ArgoCD, and Docker.

• Expertise in analytics data modeling, metric definitions, and automated monitoring/data-quality controls.

• Practical experience in managing production data systems, including incident management, root-cause analysis, runbooks, and reliability improvements.

• Comfortable collaborating with cross-functional analytics stakeholders utilizing Jira/Confluence and Agile methodologies.

• Familiarity with data security protocols, including PII protection, encryption, and access management.

• Experience with BI tools that support per-user, policy-based data access is advantageous.

• Knowledge of policy-based access control and identity management platforms is a plus.

• Awareness of multi-region data residency considerations is beneficial.

• Must reside in and be legally authorized to work in the United States.

• Required to have a minimum of 5 years of hands-on experience in data engineering, analytics engineering, or data platform engineering.

• Must have practical experience with graph databases in production and their integration with data warehouses or lakes.

• Must have direct experience in building and maintaining data pipelines using streaming/CDC technologies.

🏝️ Benefits

• Medical, Dental, and Vision Insurance through various PPO options; employer covers 90% of employee premiums and 60% of spouse/dependent premiums.

• Company-sponsored 401k plan with up to 4% matching for US employees.

• Incentive Stock Options.

• Full paid parental leave for 100% of the duration.

• Unlimited Paid Time Off (PTO).

• 12 paid company holidays annually.

• Performance-based bonuses.

• Office stipend provided.

• No-meeting Fridays.

• Flexible time off policies.

• Culture committee, coffee chats, and interest groups foster connection.

• Stipends for work-from-home arrangements.

• Occasional travel for business purposes.