Head of Data Engineering

This is a fully remote position, open to applicants in New Jersey.

📋 Description

• Develop and enhance the comprehensive data platform strategy encompassing ingestion, transformation, storage, and serving, centered around Databricks (Lakehouse, Delta, Unity Catalog) and PostgreSQL, ensuring production-grade reliability and cost efficiency.

• Create an AI-first data engineering practice that incorporates standard patterns, SDKs, and established pathways utilizing AI to expedite pipeline development, testing, documentation, and operations across various teams.

• Implement enterprise-level data governance and security measures, including cataloging, lineage, access controls, data quality, handling of PII, and enforcement of policies throughout the platform.

• Build and manage high-performing data engineering teams by recruiting, mentoring, and establishing technical standards, while driving measurable enhancements in delivery speed and platform trust.

• Enhance the broader Engineering and Data organizations by sharing reusable components, best practices, and self-service options that minimize bottlenecks and reduce reliance on vendors.

• Act as the accountable leader for essential data initiatives, overseeing the process from requirements gathering to architecture, implementation, launch, and post-launch evaluations.

• Design scalable and reliable data pipelines and platform services on Databricks and PostgreSQL, accommodating both batch and streaming workloads across marketing, sales, and servicing areas.

• Define and implement an AI-first engineering workflow that utilizes AI coding assistants, agentic tools, and automated evaluation/QA gates to enhance data engineering results without sacrificing quality or security.

• Establish standards for data governance, including Unity Catalog (or similar), lineage, data contracts, freshness and quality SLAs, and asset lifecycle management.

• Collaborate with Security and Compliance to manage the data security posture, focusing on role-based access, audit trails, encryption, PII/PHI management, and adherence to regulations pertinent to insurance data.

• Set engineering standards, review designs, pull requests, data models, and architecture, and promote adoption through thorough documentation and enablement efforts.

• Lead the evaluation of vendors and tools, providing recommendations for build/buy/insource decisions that align with unit economics, reliability, and intellectual property strategy.

• Recruit, mentor, and cultivate engineers, conduct tech talks, and foster a culture of ownership, experimentation, and continuous improvement.

⛳️ Requirements

• A minimum of 10 years in data engineering or related distributed systems, with at least 4 years in a leadership role building data engineering teams.

• Proven, hands-on experience in utilizing AI to enhance data engineering outcomes (e.g., AI coding assistants, agentic tools, LLM-assisted pipeline development, testing, or operations).

• Extensive expertise with Databricks or a comparable platform (Lakehouse, Delta Lake, Spark, Unity Catalog) and PostgreSQL in a production environment.

• Demonstrated responsibility for data governance and security programs, including cataloging, lineage, access control, data quality, and PII management.

• Strong foundation in engineering principles, particularly in Python and SQL; experience in designing scalable ETL/ELT and streaming architectures.

• A track record of establishing technical standards and successfully delivering complex data initiatives from architecture to launch.

• Outstanding communication and mentoring skills, effective with stakeholders across both technical and non-technical areas.

• A Bachelor's degree in Computer Science or a related field is required.

🏝️ Benefits

• Medical (including prescription coverage)

• Dental

• Vision

• Health Savings Account

• Health Care and Dependent Care Flexible Spending Accounts

• Group Accident

• Group Critical Illness

• Life Insurance

• AD&D

• Group Legal

• Identity Theft Protection

• Wellbeing Program and Work/Life Resources (including Employee Assistance Program)

• Paid Holidays

• Annual Paid Time Off (includes state/local paid leave where required)

• Short-Term Disability

• Long-Term Disability

• Other Leaves (e.g., Bereavement, FMLA, ADA, Jury Duty, Military Leave, and Parental and Adoption Leave)

• Savings Plan with annual nonelective company contribution.