Data Engineer

This is a fully remote position, open to applicants in District of Columbia, +1 more state.

📋 Description

• Design, construct, and sustain dependable, scalable ETL/ELT pipelines.

• Create Apache Airflow DAGs to coordinate pipeline tasks, schedules, dependencies, and data-processing workflows.

• Integrate data from databases, APIs, and files while performing transformations and maintaining data consistency.

• Develop SQL transformations and Snowflake data structures.

• Create reusable pipeline components and automate recurring processes.

• Assist in configuring and deploying pipelines across development, testing, and production environments.

• Develop and manage Snowflake schemas, tables, and views.

• Implement data loading and transformation processes in Snowflake.

• Optimize SQL queries and Snowflake workloads for enhanced performance, warehouse utilization, and cost efficiency.

• Utilize Snowflake data modeling and performance optimization techniques, including clustering when advantageous.

• Establish and maintain Snowflake roles and permissions.

• Set up Airflow schedules, dependencies, retries, timeouts, and failure handling protocols.

• Monitor workflow executions, investigate failures, and take corrective actions as necessary.

• Configure alerts and operational logging systems.

• Create Python scripts and automation for pipeline operations.

• Maintain workflow and operational documentation.

• Develop automated data validation checks and testing procedures.

• Investigate and resolve issues related to data quality.

• Adhere to data governance, privacy, security, and access management standards.

• Document data lineage and transformation logic.

• Follow procedures for error handling and data recovery.

• Collaborate with technical teams and stakeholders to clarify data requirements and deliver engineering solutions.

• Engage in code reviews, technical design discussions, and development planning.

• Contribute to version control, automated testing, and CI/CD practices.

• Suggest enhancements for reliability, scalability, maintainability, and cost efficiency.

• Share knowledge and provide technical guidance.

• Perform other related duties as assigned.

⛳️ Requirements

• Bachelor's degree in Computer Science, Data Engineering, Information Systems, Software Engineering, or an associate degree with two years of relevant experience.

• A minimum of 3 years of professional experience in data engineering, data integration, ETL/ELT development, or a related technical field.

• At least 3 years of hands-on experience with Snowflake and Apache Airflow in data engineering or data pipeline development.

• Strong expertise in SQL, including query development, data transformation, and performance optimization.

• Experience in developing, scheduling, monitoring, and troubleshooting data pipelines using Apache Airflow.

• Experience in designing and maintaining data structures, transformations, and workflows within Snowflake.

• Proficiency in Python or a similar scripting language for automation and data processing.

• Understanding of relational databases, data modeling, data integration patterns, and data quality practices.

• Familiarity with version control tools such as Git and collaborative software development practices.

• Strong analytical, troubleshooting, and problem-solving abilities.

• Capability to work independently on assigned engineering tasks while effectively collaborating with technical teams and stakeholders.

• Must reside in the Continental US.

• Must be either a US citizen or a Permanent Resident.

• May be required to possess or obtain a Public Trust designation or security clearance, depending on agency requirements.

• Successful completion of a background check, background investigation, and standard onboarding process is mandatory.

• Individuals residing in HUBZone-certified areas are strongly encouraged to apply.

🏝️ Benefits

• Competitive salary and comprehensive benefits packages.

• Health care insurance.

• Dental insurance.

• Vision insurance.

• Life insurance.

• 401(k) retirement plan.

• Paid time off, including PTO, holidays, and other legally mandated paid leave.

• Remote-first work environment.

• Equal opportunity employer.