Quality Engineer, Data

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop and execute strategies for data quality, including standards, testing practices, documentation, and maintenance processes for data pipelines and analytical datasets.

• Design and perform both automated and manual tests to ensure data accuracy, completeness, integrity, uniqueness, schema consistency, business rules, freshness, and statistical validity.

• Create automated quality checks and integration/end-to-end tests utilizing Python, PySpark/Spark, SQL, and Apache Airflow.

• Validate data transformations and pipelines across platforms such as Apache Spark, Python, AWS, Amazon S3, and other related AWS data services.

• Build reusable testing frameworks and utilities for data pipelines and CI/CD processes, ensuring that both code and data are validated prior to production release.

• Implement quality validation across various data types including raw, cleaned, curated, and analytics-ready data, establishing thresholds, rules, and acceptance criteria.

• Investigate data anomalies, schema alterations, missing data, pipeline failures, and other quality concerns; identify root causes and collaborate with Data Engineers for resolution.

• Conduct code and product reviews, ensuring that new pipelines and transformations integrate appropriate unit, integration, and data quality testing.

• Monitor data quality metrics and dashboards while supporting automated regression testing.

• Work alongside Data Engineers, Data Scientists, Analysts, and stakeholders to convert business and regulatory requirements into data quality controls throughout the SDLC.

• Assist with UAT, release validation, production deployments, and documentation of test strategies, requirements, defects, and validation procedures.

⛳️ Requirements

• Candidates MUST BE US Citizens and able to obtain Public Trust clearance.

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent experience).

• Over 7 years of professional experience in software quality assurance, testing, or quality engineering roles.

• Proficient programming skills in Python and experience in developing automated tests and test frameworks (e.g., Pytest or similar).

• Strong grasp of data quality principles and methodologies, including accuracy, completeness, consistency, uniqueness, validity, and timeliness.

• Experience with relational databases and SQL.

• Background in testing data pipelines, ETL/ELT processes, or large-scale data transformations.

• Familiarity with distributed data processing technologies such as Apache Spark/PySpark.

• Knowledge of Apache Airflow or another workflow orchestration platform.

• Experience with cloud-based data platforms, preferably AWS.

• Proven ability in creating, executing, and maintaining both automated and manual test plans and test cases.

• Experience in identifying, documenting, prioritizing, and tracking software defects.

• Background in supporting Agile software development teams.

• Excellent analytical, communication, and problem-solving abilities.

• Capability to investigate data discrepancies, determine root causes, and effectively communicate technical data quality issues to both technical and non-technical stakeholders.

🏝️ Benefits

• Remote Work (Hybrid roles will be specified in the job post).

• Competitive Compensation Package.

• Medical, Dental, and Vision coverage.

• Life Insurance, Short/Long Term Disability options.

• Employee Assistance Program.

• 401(k) plan with 4% matching.

• Generous PTO vacation policy.

• Annual Continuing Education allowance.

• Annual Wellness Budget.

• Bonus Incentive Programs (including employee referrals and performance-based rewards).