Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Create, implement, and manage test plans, scenarios, and cases for enterprise data platforms, ETL/ELT pipelines, and data integration solutions.

• Conduct comprehensive ETL testing to ensure data accuracy, completeness, transformation logic, reconciliation, and data quality across both source and target systems.

• Design and execute intricate SQL queries for backend validation and data testing.

• Evaluate large-scale data processing solutions utilizing Databricks, Apache Spark, PySpark, Python, Hive, Hadoop/HDFS, and Azure Data Factory (ADF).

• Carry out functional, integration, regression, system, and data quality testing throughout data pipelines and applications.

• Confirm batch processing, job scheduling, dependencies, and workflow execution using Control M and Unix/Linux environments.

• Detect, document, track, and re-evaluate defects while collaborating with developers, data engineers, business analysts, and other stakeholders until resolution.

• Create and sustain reusable test scripts and automation solutions as applicable.

• Engage in Agile/Scrum activities, including sprint planning, refinement, daily stand-ups, reviews, and retrospectives.

⛳️ Requirements

• Over 8 years of experience in software quality engineering/testing.

• Extensive experience in ETL testing, data validation, integration testing, and regression testing.

• Proficient in SQL, including the ability to write complex queries for source-to-target validation, reconciliation, and data quality assessment.

• Experience with large-scale data platforms and ETL/ELT pipelines.

• Practical knowledge of Databricks, Apache Spark, PySpark, Python, and Azure Data Factory (ADF).

• Familiarity with the Hadoop ecosystem including Hadoop/HDFS and Hive.

• Experience in Unix/Linux environments and with enterprise job scheduling tools such as Control M.

• Strong comprehension of data quality concepts, data transformations, data mappings, and source-to-target validation.

• Knowledge of defect management and test management processes in Agile/Scrum delivery environments.

• Excellent analytical, troubleshooting, communication, and collaboration abilities.

• Capability to work remotely from any location within the USA.

• Successful completion of a background investigation is required.

🏝️ Benefits

• Competitive salary along with a profit participation program.

• Extensive medical, dental, and vision coverage.

• Basic life insurance and accidental death & dismemberment coverage.

• 401(k) plan with matching contributions.

• CGI share purchase plan.

• Flexible and paid accrued vacation leave, ranging from 10 to 20 days annually.

• 10 paid holidays each year.

• A minimum of 80 consecutive hours of paid sick/safe leave.

• Paid parental leave, spanning from 20 to 70 consecutive business days.

• Bereavement leave, varying from 1 to 7 days per year.

• Paid jury duty leave, covering the duration summoned.

• Opportunities for learning and tuition assistance.

• Wellness and well-being programs.

• Hybrid work model providing flexibility.

• Employee resource groups and community initiatives.

• Leadership mentoring and professional development opportunities.