Senior Data Platform Engineer

This is a fully remote position, open to applicants in Pennsylvania.

📋 Description

• Design and manage distributed storage systems for the scheduling and execution of large-scale batch workloads.

• Develop and sustain an open-source, contemporary data platform.

• Enhance the utilization of storage resources while improving the reliability and fault tolerance of large-scale storage systems and data platform components.

• Partner with teams throughout the organization to comprehend workload requirements and enhance platform capabilities.

• Contribute to platform tools, automation processes, and CI/CD workflows.

⛳️ Requirements

• Over 7 years of experience in building and managing distributed storage systems or modern data platforms.

• Experience with streaming platforms such as Kafka or Pulsar.

• Proficient in Python and SQL, with experience in developing and maintaining highly available data applications using Trino and Apache Spark.

• Familiarity with table formats (Iceberg, Delta Lake, Hudi, Xtable).

• Experience in operating and optimizing at least one RDBMS (Postgres, MySQL).

• Strong debugging and problem-solving abilities in complex distributed systems.

• Capability to collaborate with various teams and communicate technical concepts effectively.

🏝️ Benefits

• Equal opportunity workplace

• Diverse teams produce the best ideas and outcomes

• Culture of inclusion, entrepreneurship, and innovation