Senior Software Engineer, Platform

This is a fully remote position, open to applicants in California.

📋 Description

• Design, develop, and maintain essential components of the streaming platform, which includes Kafka-compatible APIs, storage solutions, metadata management, and distributed services.

• Tackle intricate engineering challenges related to distributed systems, storage, networking, and performance enhancement.

• Enhance reliability through profiling, benchmarking, and thorough testing.

• Diagnose and resolve production issues within large-scale distributed environments.

• Collaborate closely with customers and field teams to comprehend real-world streaming scenarios and provide effective technical solutions.

• Engage in open-source projects to enhance the developer experience of streaming platforms.

• Participate in a rotating on-call schedule, one week per month, addressing customer tickets.

• Build dependable infrastructure that supports real-time data applications for developers and enterprises globally, as part of the Platform Engineering team.

⛳️ Requirements

• Over 5 years of experience in developing large-scale distributed systems, backend infrastructure, or cloud services.

• In-depth knowledge of Apache Kafka, Apache Pulsar, or similar distributed messaging systems.

• Familiarity with distributed systems principles, including replication, consistency, fault tolerance, scalability, and distributed storage.

• Understanding of Kafka internals, encompassing the Kafka protocol, broker architecture, KRaft, tiered storage, log storage, and compaction.

• Proven experience in creating cloud-native streaming or storage systems.

• Proficient programming skills in Java, Scala, or other systems programming languages.

• Background in designing and managing reliable production systems.

• Strong skills in debugging and performance optimization.

• Capability to independently manage projects from design to implementation and production deployment.

• Effective communication abilities and the capacity to collaborate successfully with cross-functional teams.

• Knowledge of object storage technologies such as S3, GCS, or Azure Blob Storage.

• Experience with lakehouse technologies, including Apache Iceberg or Parquet.

• Familiarity with Kubernetes and cloud-native infrastructure.

🏝️ Benefits

• Comprehensive health, dental, and vision insurance.

• Flexible work hours and remote work options.

• Opportunities for professional development and growth.

• Collaborative and innovative work environment.

• Competitive salary and performance-based bonuses.