ETL QA

This is a fully remote position, open to applicants in United States.

📋 Description

• Validate ETL pipelines during the integration and consolidation of 30 diverse databases into a unified MySQL instance.

• Develop intricate SQL queries to verify data extraction, transformation, and loading from legacy SQL databases.

• Design and implement automated reconciliation scripts to compare millions of rows between source and target systems.

• Ensure data integrity, compliance with business rules, and guarantee zero data loss or corruption throughout the migration process.

• Construct and maintain automated testing frameworks for data pipelines.

• Confirm that migrated data is accurately reflected through APIs and Data Access Layers in collaboration with C# and Java developers.

• Identify, document, and resolve issues related to data drift, truncation errors, mapping failures, and performance bottlenecks.

⛳️ Requirements

• Over 7 years of experience in crafting advanced SQL queries.

• Proficient in advanced SQL concepts including joins, CTEs, window functions, and stored procedures.

• Experience with MySQL and SQL Server.

• Demonstrated experience in testing large-scale database consolidations, data warehouse migrations, or complex ETL processes.

• Practical experience in creating automated data validation scripts using Python, C#, or specialized data testing tools like Great Expectations, dbt tests, or QuerySurge.

• Strong grasp of database schemas, primary/foreign key relationships, indexing, and data mapping documentation.

• Ability to establish data equivalence through automated reconciliation rather than relying solely on manual spot-checks.

• Nice to have: Familiarity with Postman, REST Assured, or similar API testing tools.

• Nice to have: Capability to read C# or Java code.

• Nice to have: Experience with Azure/AWS data services and CI/CD pipelines.

🏝️ Benefits

• Remote work arrangement.