This is a fully remote position, open to applicants in Virginia.

📋 Description

• Design, develop, and sustain robust data pipelines and ETL processes that facilitate seamless integration, cleansing, harmonization, and transformations within large-scale data processing systems and data lakes/data warehouses utilizing GCP native services.

• Implement performance optimization techniques such as partitioning, broadcast joins, caching, and shuffle optimization.

• Integrate data from diverse sources including databases, APIs, cloud storage, and potentially streaming sources.

• Transition legacy data stores and ETL pipelines to modern, cloud-native architectures on GCP.

• Create scalable batch and real-time streaming data pipelines utilizing Python, SQL, and Apache Beam.

• Enforce data governance, establish data quality checks, and implement secure data access controls in accordance with stringent federal security regulations.

• Collaborate with data scientists, cloud infrastructure engineers, and mission stakeholders to enhance data models and improve query performance.

• Design and execute CI/CD pipelines to optimize the build, test, and deployment workflows.

⛳️ Requirements

• 3 to 5+ years of experience in data engineering, data warehousing, and/or big data processing.

• 2+ years of practical experience in constructing data pipelines and/or data solutions on Google Cloud Platform (BigQuery, Dataflow, and/or Cloud Composer).

• Proficient programming skills in Python along with advanced SQL capabilities.

• Familiarity with ETL/ELT concepts and associated tools.