Data Engineer II

This is a fully remote position, open to applicants in New York.

📋 Description

• Develop ingestion pipelines and Airflow DAGs based on designs provided by senior engineers, ensuring to include code, unit tests, and comprehensive documentation.

• Assist with data security and governance initiatives, which include PII masking and implementing access controls.

• Contribute to reverse ETL processes and other data delivery tasks under the guidance of senior engineers.

• Enhance existing pipelines and incorporate feedback from reviews.

• Manage on-call responses for pipeline failures, adhere to runbooks, and escalate issues with detailed context.

• Collaborate with senior engineers to address data integrity challenges.

• Compose pull request descriptions that are easy for reviewers to understand and pose clarifying queries when necessary.

• Identify and communicate blockers at an early stage.

• Foster relationships with BI analysts, data engineers, data scientists, and other internal stakeholders.

• Collect and clarify project requirements.

• Engage in and facilitate code and system inspections.

• Assist in defining and adhering to best practices in data engineering.

• Mentor junior data engineers as the role evolves.

⛳️ Requirements

• 1–3 years of experience in professional software or data engineering.

• Strong skills in gathering, evaluating, and analyzing requirements.

• Solid proficiency in Python programming.

• In-depth knowledge of SQL and ETL/ELT processes for complex data transformations.

• Capability to read and write unit tests and navigate an established codebase and its conventions.

• Familiarity with or a strong willingness to learn Airflow / Managed Workflows for Apache Airflow quickly.

• Basic understanding of ingestion, idempotency, scheduling, and data quality principles.

• Knowledge of various technologies including Snowflake, Databricks, AWS, dbt, Tableau, MongoDB, and PostgreSQL.

• Familiarity with Git-based version control and the pull-request code review process.

• Excellent communication skills, including the ability to ask clarifying questions, write clear PR descriptions, and escalate blockers with useful context.

• A growth mindset with the ability to incorporate review feedback and advocate for best practices.

🏝️ Benefits

• Competitive base salary and performance-based bonus.

• Detailed benefits information can be found at https://benefitsatfanatics.com/.

• Opportunity for an in-person onboarding experience through "Launching into Better: LIVE" for full-time, non-seasonal hires.

• Company-sponsored or supported travel/participation for relevant onsite interviews or onboarding sessions, subject to role and location requirements.