Senior Data Platform Engineer – Kafka, PostgreSQL

This is a fully remote position, open to applicants in Kazakhstan.

📋 Description

• Design, implement, and manage highly available PostgreSQL configurations utilizing CloudNativePG on Kubernetes and Patroni-style setups on bare metal.

• Create, deploy, and maintain distributed Apache Kafka clusters with Strimzi.

• Develop bespoke enterprise-level open-source Kafka clients, standalone connector services, and utility libraries, preferably using Go.

• Establish integrations that link PostgreSQL authentication and row-level security policies with Active Directory, LDAP, and OIDC identity providers.

• Scale high-throughput CDC pipelines leveraging Debezium and Kafka Connect.

• Build resilient architectures that ensure data integrity across databases and topics while mitigating dual-write risks.

• Design and manage cross-region failover and disaster recovery orchestration for both PostgreSQL and Kafka.

• Define replication architectures, mechanisms to prevent split-brain scenarios, and RPO/RTO objectives for region or cluster failures.

• Optimize PostgreSQL for high-volume ingestion and operations with zero downtime.

• Tune WAL, logical replication streams, connection pooling with PgBouncer, CSI storage volumes, and CNI network paths.

• Manage declarative infrastructure-as-code utilizing Terraform and ArgoCD.

• Create self-service workflows for the provisioning of databases, topics, schemas, and ACLs.

• Implement multi-tenant isolation using database-level RLS, CNI network policies, mTLS, and Kafka topic-level RBAC.

• Take ownership of the event-streaming, transactional database, and custom connectivity framework for the k0rdent-ai platform.

⛳️ Requirements

• Over 10 years of experience in systems, software, data, or platform engineering.

• Proven track record of managing large-scale, mission-critical data infrastructures.

• Proficiency in Go (Golang) or similar programming languages; capable of writing clean, concurrent systems code and custom client libraries.

• More than 5 years of experience managing native PostgreSQL.

• In-depth knowledge of PostgreSQL internals, including WAL streams, logical replication, publications/subscriptions, and connection architectures.

• Practical experience operating PostgreSQL on Kubernetes through a mature Postgres operator; CloudNativePG is preferred or an equivalent.

• Extensive experience managing Apache Kafka at scale on Kubernetes via a declarative operator; Strimzi is preferred or an equivalent.

• Demonstrated experience in optimizing Kafka Connect and Debezium pipelines.

• Experience in writing or contributing to open-source database connectors, Kafka client libraries, or integration frameworks.

• Hands-on experience with integrating distributed systems and enterprise IAM/SSO providers such as Active Directory, LDAP, Okta, Azure AD, or OIDC.

• Strong understanding of Kubernetes primitives, including Pod lifecycles, Operators, CSI storage layers, and CNI overlay routing.

• Production experience in AWS or GCP.

• Bachelor's degree in Computer Science & Engineering or a related field, or 10+ years of equivalent relevant experience.

• Nice to have: experience with cross-region disaster recovery and failover orchestration.

• Nice to have: contributions to open-source Kafka connectors, PostgreSQL operators, or Go-based database libraries.

• Nice to have: familiarity with schema governance, Apache Flink or Kafka Streams, SOC 2/ISO 27001, Redis, and MongoDB.

🏝️ Benefits

• Opportunities for professional development and training.

• Participation in conferences and working groups.

• Company outings, happy hours, hackathons, and tech talks.

• Competitive compensation package along with a robust benefits plan.

• Chance to collaborate with passionate, talented colleagues and Fortune 500 and Global 2000 clients.

• Environment that fosters open-source innovation.

• Continuous opportunities for growth.