Principal Data Engineer, Data Platform

This is a fully remote position, open to applicants in United States.

📋 Description

• Define and advance the technical vision and roadmap for APFM's Databricks platform and the wider data ecosystem.

• Propel the development of APFM's federated data operating model, striking a balance between domain ownership and enterprise governance and standards.

• Collaborate with Analytics Engineering and business stakeholders to refine the semantic layer, ensuring that business definitions, metrics, and dimensions remain consistent, discoverable, and reliable.

• Set standards for metadata management, lineage, discoverability, stewardship, and data quality throughout the organization.

• Foster the creation of reusable data products and shared platform capabilities that facilitate self-service analytics at scale.

• Lead the adoption and governance of Databricks features including Delta Lake, Unity Catalog, Workflows, AI/BI Genie, MLflow, Vector Search, and associated platform services.

• Work closely with AI and Machine Learning teams to establish scalable operational patterns for model development, deployment, monitoring, and governance.

• Guide technical decision-making across various teams and domains, ensuring alignment of platform investments with business outcomes.

• Mentor engineers across the organization, helping to enhance technical skills and promote engineering excellence.

⛳️ Requirements

• Over 10 years of experience in data engineering, data platform architecture, governance, or related fields.

• In-depth expertise in designing and managing enterprise-scale Databricks Lakehouse platforms.

• Extensive experience with Spark, Delta Lake, Unity Catalog, Databricks Workflows, and platform governance functionalities.

• Strong command of SQL, Python, and PySpark, with a background in building and optimizing data processing workloads on Databricks.

• Experience in designing and supporting semantic layers, metrics governance, and self-service analytics functionalities.

• Solid understanding of governance, metadata management, lineage, observability, security, and reliability engineering principles.

🏝️ Benefits

• 401(k) plus match

• Dental insurance

• Health insurance

• Vision Insurance

• Paid Time Off