Senior Data Engineer – ETL, Pipelines

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and maintain ETL and streaming pipelines that process high-volume event and payment data.

• Provide dependable and precise data to the analytics warehouse.

• Collaborate with the Lead Data Analyst to establish warehouse schemas for creator earnings, payment throughput, retention, lifetime value, and cohort behavior.

• Create idempotent, replayable, and observable jobs that are resilient to traffic spikes, retries, and partial failures.

• Manage high-volume events using BullMQ, Redis, AWS SQS, and RabbitMQ.

• Instrument pipelines with OpenTelemetry.

• Implement reconciliation checks, deduplication, backfills, and the processing of late or out-of-order events.

• Work alongside payments and platform teams to ensure every transaction is captured accurately, only once, and in a timely manner.

⛳️ Requirements

• Approximately 5–10 years of relevant engineering experience.

• Extensive experience with batch and/or streaming data pipelines.

• In-depth knowledge of event-driven architectures and message queues.

• Strong understanding of idempotency, at-least-once versus exactly-once processing, retries, and out-of-order event delivery.

• Professional experience with TypeScript and Node.js.

• Proficient in SQL and data modeling.

• Capable of collaborating with analysts to design effective and intuitive business-query schemas.

• Strong emphasis on data integrity, especially concerning payments and financial transactions.

• Experience in implementing reconciliation, auditing, monitoring, and data-quality controls.

• Previous experience in a high-volume B2C environment such as gaming, gambling, fintech, payments, or other real-money platforms.

• Experience with .NET or C# is considered advantageous.

• Familiarity with payment or transaction data and financial reconciliation.

• Experience with MongoDB change streams or change data capture.

• Knowledge of data warehouse tooling and dbt-style modeling.

• Experience working with Kubernetes and infrastructure as code.

🏝️ Benefits

• Direct-hire opportunity; successful candidates will be employed or engaged directly by the client.

• Recruitment and selection process managed by 2am.tech.