Senior Software Engineer – Stream Storage, Apache Flink

This is a fully remote position, open to applicants in Spain.

📋 Description

• About the Role: We are developing the next generation of stream-native storage systems focused on table semantics, rather than merely logs. Our Stream Storage team is responsible for creating high-performance distributed systems that enable real-time analytics, streaming pipelines, and transactional workloads.

• As a Senior Software Engineer, you will play a key role in enhancing core stream storage infrastructure, Apache Fluss (open-source), and internal production-grade systems that utilize Fluss.

• This position operates at the crossroads of distributed systems, storage engines, streaming systems, and database internals.

• What You’ll Do: Core Engineering

• - Design and develop distributed storage components for streaming tables.

• - Manage table lifecycle aspects such as schema evolution, ingestion, compaction, retention, and indexing.

• - Improve support within the Fluss Lakehouse ecosystem.

• - Contribute new features and bug fixes to the Apache Fluss OSS project.

• - Engage in design discussions and Fluss Improvement Proposals (FIPs).

• - Enhance the performance and reliability of the Fluss table engine.

• - Improve integration with streaming engines (e.g., Flink).

• - Interact with the open-source community through PR reviews and discussions.

• Internal Platform Development

• - Develop internal tools and services built on top of Fluss.

• - Boost observability through metrics, logging, and failure diagnostics.

• - Optimize production deployments.

• - Contribute to benchmarking and testing frameworks.

⛳️ Requirements

• What We’re Looking For: Required Experience

• - 5+ years of experience in building distributed systems or storage systems.

• - Strong proficiency in Java, Go, Rust, or comparable systems languages.

• - Solid understanding of:

• - Consensus and replication techniques (Raft, Paxos, etc.).

• - Storage engines (LSM trees, B-Trees, WAL, compaction).

• - Streaming systems (Kafka, Pulsar, Flink, etc.).

• - Transactional systems and consistency models.

• - Experience in debugging production distributed systems.

• Strongly Preferred

• - Background in database internals or stream processing engines.

• - Familiarity with table formats (Iceberg, Hudi, Delta, etc.).

• - Contributions to open-source projects, especially under the ASF.

• - Experience working with Flink or streaming SQL engines.

🏝️ Benefits

• Why Join Us

• - Work on table-first stream storage, moving beyond just message logs.

• - Have a direct impact on the evolution of Apache Fluss as an open-source project.

• - Tackle complex challenges related to consistency and performance trade-offs.

• - Influence architectural decisions in a rapidly evolving field.

• - Collaborate with engineers who are passionate about distributed systems.