This is a fully remote position, open to applicants in United States, +1 more country.

📋 Description

• Construct platform engineering infrastructure that empowers data scientists, clinicians, engineers, and agents to securely modify data.

• Create tracking infrastructure for product analytics, experimentation, and feature gating.

• Enforce event schemas from the source.

• Develop Databricks Bronze, Silver, and Gold data processing infrastructure.

• Execute automated schema evolution, contract tests, safe backfills, freshness monitoring, and volume monitoring.

• Establish ML infrastructure for feature computation and serving, training and evaluation pipelines, and integration of model outputs into products.

• Create self-service templates, local development and preview environments, policy-as-code for PHI, ownership routing for alerts, and progressive deployment gates.

• Implement declarative contracts, data-focused CI diffs, validation gates, lineage tracking, and one-command rollback functionalities.

⛳️ Requirements

• Developed an internal platform or infrastructure that was adopted by other engineers.

• Managed production data or ML systems, including on-call responsibilities and incident resolution.

• Proficient in Python and SQL.

• Familiarity with Databricks or similar lakehouse platforms such as Snowflake or BigQuery.

• Designed and evolved interfaces and schemas without disrupting dependent teams.

• Experience with testing and CI processes for data or ML systems.

• Minimum of 8 years of engineering experience.

• Nice to have: experience with dbt, DLT, Dagster, Airflow, Kafka, Spark Structured Streaming, data contracts, data diffing, lineage tools, Terraform, feature stores, MLOps and evaluation tools, agentic coding workflows, healthcare, PHI, or HIPAA.