Platform Engineer – Data, SDE-3/Staff

This is a fully remote position, open to applicants in India.

📋 Description

• Develop and enhance open-lakehouse foundations utilizing Apache Iceberg, Delta Lake, or Apache Hudi.

• Design and manage batch and streaming ingestion systems, encompassing CDC, data contracts, deduplication, checkpointing, replay, recovery, and failure management.

• Create platform services, APIs, and self-service functionalities for engineering and product teams.

• Construct and oversee orchestration and workload execution systems across Kubernetes, along with technologies such as Prefect, Argo, or Airflow.

• Implement capabilities for metadata, lineage, governance, data quality, and observability.

• Ensure production reliability and operational excellence, which includes incident response, SLAs, data freshness, performance optimization, cost management, capacity planning, and recovery enhancements.

• Define engineering standards and architectural patterns to facilitate scaling across teams and workloads.

⛳️ Requirements

• Over 7 years of experience in software or platform engineering, showcasing SDE-3 / Staff-level ownership and impact.

• Practical experience in managing a shared data platform in a production environment.

• Strong knowledge in distributed systems design and production-quality software development using Python, Java, Go, or Scala.

• Familiarity with batch processing, streaming, data ingestion/CDC, storage and query systems, and platform operations.

• Capability to enhance reliability, scalability, performance, cost efficiency, or developer self-service in large-scale production settings.

• A strong individual contributor who can influence architecture, set technical standards, and lead intricate engineering projects.

• Responsible for end-to-end ownership from architecture and implementation to deployment, monitoring, incident response, and continuous enhancement.

• Experience with Apache Iceberg, Apache Hudi, Delta Lake, Apache Kafka, Apache Flink, Debezium, Apache Spark, Trino, Pinot, Kubernetes, cloud-native workload platforms, data catalogs, metadata platforms, lineage systems, governance frameworks, data quality platforms, or observability platforms.

• Essential skills include: Data Platform; Distributed Systems.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Flexible work hours and remote work options.

• Opportunities for professional development and continuous learning.

• Collaborative and innovative work environment.