Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Refactor and transfer existing ETL/ELT and analytics pipelines to a new environment.

• Migrate underlying data and ensure correct landing of migrated data.

• Collaborate with a Staff Data Engineer to comprehend the migration plan, sequence tasks, and provide regular updates on progress.

• Revise PySpark, DABs, and dbt ETL/ELT code to align with new account patterns and standards.

• Migrate data pipelines and datasets, including backfills, and confirm source-target consistency.

• Establish and execute reconciliation checks for row counts, schema adherence, data freshness, and completeness.

• Transfer dashboards and downstream reporting assets while coordinating with analysts.

• Update orchestration, scheduling, alerting, and runbooks for supportability after cutover.

• Address pipeline failures and data inconsistencies during and following cutover.

• Document migrations, modifications, and ownership details.

• Manage related migration and data engineering requests as the program progresses.

⛳️ Requirements

• Over 6 years of experience in Data Engineering, Analytics Engineering, or Software Engineering with production data systems.

• Proficient in SQL and Python for developing scalable data pipelines and transformations.

• Familiarity with AI tools such as Claude Code and Devin for enhancing software development processes.

• Practical experience in constructing ELT pipelines using modern cloud data platforms.

• Experience with Databricks is mandatory.

• Proven ability to develop robust data-migration testing frameworks for completeness and quality assurance.

• Extensive experience with dbt, encompassing model development, testing, documentation, and CI/CD integration.

• Expertise in designing and operating production-grade data pipelines with monitoring and observability capabilities.

• Strong collaboration abilities, capable of partnering with engineering teams, analysts, and operational stakeholders.

• A Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics, or a related field, or equivalent practical experience.

• Previous experience in data or platform migration is essential.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Flexible work hours and remote work options.

• Opportunities for professional development and continuing education.

• A collaborative and innovative work environment.