Data Engineering Manager

This is a fully remote position, open to applicants in New York, +1 more state.

📋 Description

• Lead a team of 2-4 Data Engineers in the design and construction of data pipelines from diverse sources to a target data warehouse, utilizing both real-time and batch loading techniques.

• Facilitate performance evaluations, offer constructive feedback, and focus on the career advancement and long-term growth of engineers.

• Take ownership of the complete data platform design and execution, encompassing ingestion, transformation, outgestion, scheduling, alerting, and monitoring.

• Convert business strategies and requirements into a technical roadmap, serving as a reliable point of contact for business, client, and technical stakeholders.

• Set standards for data governance, documentation, data quality, pipeline observability, lineage, access control, metadata management, and integration workflows.

• Enhance platform reliability, scalability, performance, cost efficiency, monitoring, and alerting.

• Lead proof-of-concept initiatives to evaluate, validate, and refine technical processes.

• Advocate for AI integration within the data stack, including agentic tooling and AI-assisted pipeline development.

• Collaborate with business stakeholders early to uncover upstream opportunities and deliver increased value.

⛳️ Requirements

• Over 3 years of experience managing a data engineering team, in addition to at least 5 years of hands-on data engineering experience prior to or concurrently with that management experience.

• More than 4 years of practical experience with Databricks, including Unity Catalog, Delta Lake, and workflow orchestration.

• At least 4 years working with cloud technologies; AWS is preferred, but Azure or GCP experience is also welcome.

• Proficient hands-on coding experience with Python and PySpark within a distributed computing environment.

• Strong foundation in code-first data transformation workflows, scalable data modeling, and best practices for data quality.

• Extensive knowledge of data governance, including standards for data quality, lineage, access control, and documentation.

• Experience with practices for pipeline observability and monitoring, including alerting, backfill management, and failure triage.

• Familiarity with GitHub and CI/CD processes in a collaborative engineering setting.

• Exceptional communication skills with the ability to convey complex technical concepts to both technical and non-technical audiences in client-facing or external partnership scenarios.

• Understanding of how AI is transforming data engineering, comfort with agentic tooling, and experience in integrating AI into the stack.

• Capacity to engage with data engineering early, comprehend organizational data usage and creation, and mobilize an existing team for enhanced delivery.

• Not open to visa sponsorship or transfer of visa sponsorship, including H1-B, F-1, OPT, STEM-OPT, or TN visa.

• Not available for corp-to-corp work.

🏝️ Benefits

• Health Insurance Coverage (medical, dental, and vision).

• Life Insurance.

• Short and Long-Term Disability Insurance.

• Flexible Spending Accounts.

• Holiday Pay.

• 401(k) with matching contributions.

• Employee Assistance Program.

• Paid Parental Bonding Benefit Program.

• Flexible Paid Time Off (PTO): Full-time employees earn 20 days of PTO for a complete calendar year, increasing to 25 days after five years of service.