Staff Data Platform Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Lead a team of engineers to execute data platform initiatives from start to finish.

• Take ownership of data architecture across OLTP, OLAP, and NoSQL databases.

• Design and manage distributed real-time and batch data pipelines utilizing Python and PySpark with Kafka, Kinesis, and SNS/SQS.

• Spearhead entity resolution and Master Data Management (MDM) architecture for provider data.

• Develop and maintain data products with well-defined contracts, SLAs, and identified consumers.

• Ensure accountability for data reliability, freshness, quality, and cost, supported by observability metrics.

• Enhance performance and scalability through profiling, benchmarking, and re-architecting as data volume increases.

• Establish data foundations that support AI/ML workloads.

• Mentor engineers, elevate standards in design and code reviews, and collaborate with product, analytics, and ML teams.

• Participate in on-call rotations and transform incidents into systemic solutions.

⛳️ Requirements

• Over 10 years of experience in developing production data systems, including roles as a Staff or Tech Lead.

• Proven experience leading a team of engineers to deliver intricate data systems.

• Proficiency in Python and extensive experience with production PySpark.

• In-depth knowledge of distributed systems, including consistency, partitioning, idempotency, backpressure, and failure management.

• Significant experience with entity resolution and MDM, including matching, deduplication, survivorship, and maintaining golden records at scale.

• Created and managed data products utilized by other teams.

• Delivered measurable enhancements in performance and scalability.

• Expertise across OLTP (Aurora, PostgreSQL, MySQL), OLAP (Redshift, Athena, Snowflake, ClickHouse), and NoSQL (MongoDB, DynamoDB, Cassandra) technologies.

• Experience in stream processing and event-driven architectures (Kafka, Kinesis, SNS/SQS).

• Practical knowledge of data observability, including quality checks, lineage, monitoring, and alerting.

• Developed data platforms that support machine learning workloads.

• Influences technical direction across teams and communicates effectively with non-technical stakeholders.

• Experience managing sensitive data (PII/PHI).

• Familiarity with AI-assisted development tools (Cursor, Claude Code, MCP).

• Understanding of Change Data Capture (CDC) patterns.

• Knowledge of data mesh or data-platform-as-a-product methodologies.

• Experience with schema registries or data catalogs.

• Proficiency with orchestration tools (AWS Glue, EMR, Managed Flink, Airflow/MWAA, dbt).

• Background in healthcare or other regulated environments.

🏝️ Benefits

• Full coverage of health, dental, and vision insurance premiums for employees.

• Unlimited Paid Time Off (PTO) for the US-based team.

• Minimum of two weeks off each year to recharge for the US-based team.

• Health insurance provided for employees in India.

• Statutory leave benefits for employees in India.

• Additional wellness leave (menstrual) for women in India.