Senior Data Platform Engineer

This is a fully remote position, open to applicants in Kazakhstan, +1 more country.

📋 Description

• Design, implement, and manage the production streaming platform utilizing Kubernetes.

• Oversee the lifecycle of the cluster, including topics, partitioning, retention strategies, capacity planning, and upgrades.

• Construct and uphold change-data-capture and connector pipelines linking operational databases, the streaming layer, and analytical storage.

• Take ownership of schema governance, which encompasses schema registry, compatibility rules, and versioning.

• Create custom connectors, transformations, and stream-processing logic using Java or Go.

• Develop a self-service onboarding system for topics, schemas, and access via infrastructure as code.

• Automate the platform using Terraform and GitOps methodologies.

• Monitor the pipeline with metrics, lag and throughput monitoring, alerting, and dashboard creation.

• Implement security measures and multi-tenant isolation throughout the streaming layer.

• Ensure message ordering, idempotency, exactly-once delivery when necessary, replay capabilities, and disaster recovery protocols.

• Collaborate with backend and SRE teams on event schema design, producer patterns, and failure modes.

• Participate in on-call support for the platform.

• Guide engineers on streaming architecture and contribute through design reviews and documentation.

⛳️ Requirements

• Over 10 years of experience in software, data, or platform engineering, with substantial hands-on responsibility for a production event-streaming platform.

• Expertise in Apache Kafka (or Confluent Platform/Cloud), including administration, tuning, troubleshooting, and capacity management at a production scale.

• Strong experience with Kafka Connect and CDC, including Debezium, JDBC, and the development of custom connectors or transformations.

• Established schema-registry practices and compatibility management across numerous independent producers and consumers.

• Proficient programming skills in Java, Python, or Go.

• In-depth knowledge of SQL and relational databases, including replication and CDC mechanisms.

• Fluency in Kubernetes, including deploying, managing, and debugging stateful applications.

• Experience with infrastructure as code using Terraform or similar tools, along with ownership of CI/CD pipelines.

• Excellent written English skills for design documentation, runbooks, and asynchronous reviews across global time zones.

• Familiarity with Apache Kafka or Confluent, Kafka Connect, Schema Registry, and stream processing.

• Experience with CDC tools such as Debezium or similar, connector development, and pipelines to analytical stores.

• Experience deploying Kubernetes, Docker, and Helm-packaged applications in production environments.

• Proficient in Terraform, CI/CD, and GitOps practices.

• Experience with PostgreSQL at scale, as well as at least one cloud data warehouse or search store.

• Knowledge of AWS or GCP, including networking, IAM, and managed Kubernetes services.

• Experience with mTLS, OIDC/SSO, RBAC, and Prometheus/Grafana monitoring.

• A Bachelor’s degree in Computer Science & Engineering or a related field, or 10 years of relevant experience.

🏝️ Benefits

• Opportunities for professional development and training.

• Participation in conferences and working groups.

• Enjoy company outings, happy hours, hackathons, and tech talks.

• Competitive compensation package complemented by a robust benefits plan.

• Flexible remote work arrangements.