This is a fully remote position, open to applicants in United States.

📋 Description

• Take charge of designing the control plane responsible for placing, reconciling, and managing VMs throughout the Compute fleet.

• Establish the database partitioning and sharding strategy aimed at enhancing control plane state management by 5x.

• Create capacity models for both placement and inventory state.

• Manage the placement reconciliation loop, which encompasses desired-state modeling, idempotent reconcilers, drift detection, and reconciliation-lag service level objectives (SLOs).

• Streamline bulk and asynchronous operations into a unified model that utilizes standard operation resources, batching, concurrency limits, and automatic halting upon failure thresholds.

• Set consistency assurances for each data path.

• Implement idempotency and fencing in controller actions.

• Define saga-style workflows.

⛳️ Requirements

• A minimum of 12 years of experience in developing distributed systems, with a focus on owning control planes or large-scale orchestration, or equivalent experience.

• Extensive database expertise in SQL, covering partitioning, sharding, replication, and online schema modifications.

• Proven experience in building asynchronous systems, which includes durable task queues, event-driven workflows, and designing for retry and backpressure.

• Practical experience in building and operating containerized applications, particularly with Kubernetes.

• Proficiency in Python or Go is required.

• Familiarity with an additional language or ColdFusion is advantageous, though not mandatory.

• In-depth understanding of consistency concepts, including leases, fencing, idempotency, and optimistic concurrency.

• Practical experience with reconciliation or controller patterns, such as those found in Kubernetes-style controllers or similar frameworks.