Director, Data Engineering

This is a fully remote position, open to applicants in Florida, +4 more states.

📋 Description

• Build, guide, and mentor a high-performing Data Engineering team.

• Set engineering standards, development methodologies, coding guidelines, and delivery frameworks.

• Encourage ownership, innovation, automation, and ongoing improvement.

• Cultivate future engineering leaders through coaching and mentorship.

• Act as the technical lead and subject matter expert for the Databricks Lakehouse Platform.

• Define enterprise standards and best practices for Databricks.

• Spearhead the adoption of Unity Catalog, Lakeflow, Delta Lake, AI/ML, Workflows, and optimize performance.

• Create reusable engineering frameworks and collaborate with Databricks product teams and strategic partners.

• Oversee the migration of legacy ETL workloads to modern Databricks ELT architectures.

• Design scalable, resilient, cloud-native data pipelines.

• Drive automation across ingestion, transformation, orchestration, deployment, monitoring, and recovery.

• Optimize performance, scalability, reliability, and manage cloud costs effectively.

• Construct dependable data pipelines that support analytics, AI, operational reporting, and executive decision-making.

• Work with Data Architecture on enterprise information models and integration patterns.

• Collaborate with Data Governance & Trust on governance, metadata, lineage, and quality assurance.

• Enable the creation of reusable enterprise data products.

• Implement CI/CD, automated testing, infrastructure as code, and DevOps best practices.

• Enhance observability, monitoring, and operational excellence.

• Lead root cause analyses and continuous improvement projects.

• Define engineering KPIs and enhance delivery performance.

⛳️ Requirements

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related discipline.

• Over 10 years of experience in enterprise Data Engineering.

• More than 5 years of experience leading Data Engineering teams in large enterprise settings.

• Expert-level experience in designing and implementing enterprise solutions with the Databricks Lakehouse Platform.

• In-depth knowledge of Apache Spark, Delta Lake, SQL, Python, PySpark, and distributed data processing.

• Strong understanding of Unity Catalog, Delta Live Tables (Lakeflow), Databricks Workflows, notebooks, cluster optimization, and platform management.

• Proven track record in modernizing enterprise data platforms and transitioning legacy ETL solutions.

• Experience with CI/CD pipeline implementation, Git-based development, Infrastructure as Code, and DevOps practices.

• Solid grasp of Medallion Architecture, dimensional modeling, and contemporary enterprise data architectures.

• Excellent communication, leadership, and stakeholder management abilities.

• A Master's degree is preferred.

• Databricks Certified Data Engineer Professional certification is preferred.

• Experience with Azure cloud services and enterprise security is preferred.

• Familiarity with Data Vault 2.0 is preferred.

• Knowledge of AI, machine learning, and enterprise analytics platforms is preferred.

• Experience in the Insurance or Financial Services industry is preferred.

• Experience leading large-scale modernization initiatives is preferred.

• Employment offers are contingent upon the successful completion of a background screening.

🏝️ Benefits

• Eligibility for discretionary annual bonuses.

• Paid time off (PTO).

• Medical insurance coverage.

• Dental insurance coverage.

• Vision insurance coverage.

• Retirement savings plan.

• Disability insurance protection.

• Life insurance coverage.