Principal Data Platform Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design and establish the organization’s data and AI platform from the ground up, encompassing architecture, compute, storage, and the lakehouse foundation.

• Implement infrastructure-as-code for the platform using Terraform.

• Create CI/CD pipelines that facilitate the progression of work from development to the accredited production environment.

• Develop data governance frameworks, including cataloging, lineage, and fine-grained access control.

• Build and manage ingestion, transformation, and pipeline layers that convert raw and synthetic data into governed, analysis-ready data products.

• Design the platform in accordance with FedRAMP Moderate, NIST 800-171, and CUI constraints.

• Define artifact promotion processes to ensure that only signed, validated artifacts enter the accredited environment.

• Collaborate with cloud engineering across the infrastructure/security boundary, taking ownership of the in-platform layer.

• Empower the data science and ML team with platform capabilities, governed data, and tools for models and AI features.

• Ensure platform reliability, performance, and cost discipline as usage increases.

• Establish engineering standards, patterns, and documentation for the expanding data team.

• Convert business requirements into efficient, well-architected data solutions.

⛳️ Requirements

• U.S. citizenship is required.

• Must be able to obtain and maintain a T5/SSBI federally adjudicated clearance; an active clearance is preferred.

• Over 8 years of experience in data engineering or data platform engineering, with proven principal-level ownership.

• Experience in building a data platform or lakehouse from the ground up, with full ownership of architecture and construction.

• Design experience in batch and, when necessary, streaming data pipelines and SQL-based transformations on a lakehouse/Delta foundation.

• Proficiency in infrastructure-as-code using Terraform and CI/CD for data workloads, including environment promotion from development to production.

• Expertise in platform-level data governance, including cataloging, lineage, and fine-grained access control.

• Practical cloud experience with a major provider; Azure is preferred, with AWS or GCP considered.

• Strong skills in Python and SQL.

• Proven ability to collaborate across infrastructure/security boundaries and establish technical standards for fellow engineers.

• Excellent analytical, troubleshooting, and communication abilities.

• Minimum of 12 years of work experience with a BS/BA degree.

• Preferred: Practical experience with Databricks, including Unity Catalog, Databricks Asset Bundles, and MLflow.

• Preferred: Familiarity with regulated or accredited environments, such as FedRAMP, NIST 800-171, CMMC, CUI handling, or the ATO/RMF process.

• Preferred: Active security clearance (T5/SSBI or higher).

• Preferred: Experience in government or defense contracting.

• Preferred: Understanding of MLOps patterns, including model registry and model serving.

• Preferred: Knowledge of cost governance/FinOps discipline for cloud data platforms.

• Preferred: Experience with Spark/PySpark.

• Ability to transform business needs into efficient, well-architected data solutions.

• Strong collaboration skills across both technical and non-technical teams.

• Clear documentation and communication in fast-paced environments.

🏝️ Benefits

• Potential eligibility for overtime.

• Potential eligibility for shift differential.

• Potential eligibility for a discretionary bonus.

• Equal opportunity employment, including for individuals with disabilities and protected veterans.