Data Engineer II

This is a fully remote position, open to applicants in New York.

📋 Description

• Design and implement ingestion pipelines and Airflow DAGs based on the specifications provided by a senior engineer, utilizing the team's established scaffolding and conventions — which includes coding, unit testing, and documentation.

• Assist with data security and governance initiatives, such as PII masking and access controls, adhering to established methodologies.

• Engage in data delivery tasks, including reverse ETL integrations, under the supervision of senior engineers.

• Modify and enhance existing pipelines by adding and extending fields, while incorporating feedback from reviews and applying learned methodologies to future projects.

• Respond to on-call alerts for pipeline failures, follow runbook procedures, and escalate issues with clear context when necessary.

• Collaborate with senior engineers on diagnosing data integrity problems that you are not yet equipped to resolve independently.

• Compose clear, reviewer-friendly pull request descriptions and seek clarification on requirements before initiating new tasks.

• Identify and communicate blockers early with contextual information rather than remaining silent when facing challenges.

• Foster strong professional relationships with internal stakeholders (such as BI analysts, other data engineers, and data scientists) to assist in gathering and clarifying requirements.

• Participate in and conduct code and system reviews.

• Assist the team in defining and adhering to best practices in data engineering.

• Mentor junior data engineers as you progress in your role.

⛳️ Requirements

• 1–3 years of experience in professional software or data engineering.

• A self-motivated learner with a strong ability to gather, evaluate, and analyze requirements effectively.

• A solid foundation in Python, along with a deep understanding of SQL and ETL/ELT processes for complex data transformations.

• Proficiency in reading and writing unit tests, as well as working within an established codebase and adhering to conventions.

• Familiarity with (or a willingness to quickly learn) workflow orchestration tools such as Airflow (Managed Workflows for Apache Airflow).

• A basic understanding of data pipeline concepts, including ingestion, idempotency, scheduling, and data quality.

• Knowledge of several technologies such as Snowflake, Databricks, AWS, dbt, Tableau, MongoDB, and PostgreSQL.

• Experience with Git-based version control and pull request-based code review processes.

• Excellent communication skills — capable of asking clarifying questions, writing clear pull request descriptions, and effectively escalating blockers with relevant context instead of remaining silent when issues arise.

• A growth mindset: receptive to review feedback, committed to improving processes, and advocates for best practices to mitigate technical debt.

🏝️ Benefits

• For information about our benefits, please visit https://benefitsatfanatics.com/