Senior Data Engineer – Real Time Streams

This is a fully remote position, open to applicants in Ukraine, +2 more countries.

📋 Description

• Act as a senior engineer within the team responsible for the real-time data platform.

• Create, implement, deploy, and manage streaming jobs, CDC pipelines, Kafka Connect bridges, and downstream data sinks.

• Design, implement, and manage stateful Apache Flink streaming pipelines utilizing keyed state, windowing, watermarks, timers, side outputs, and custom sources/sinks.

• Develop and sustain CDC pipelines with Flink CDC or Debezium, handling snapshot/incremental processes, schema evolution, and ensuring downstream idempotency.

• Construct and maintain Kafka and Kafka Connect pipelines, focusing on topic and partition design, distributed source/sink connectors, schema and converter management, and dead-letter routing.

• Oversee the Kafka → Flink → Postgres / TimescaleDB / ClickHouse / BigQuery data flow, including schema design, idempotency strategy, batch tuning, and observability.

• Identify and resolve production issues such as checkpoint failures, backpressure, state growth, sink slowness, autoscaler oscillation, and restart loops.

• Strengthen the platform by ensuring delivery guarantees, managing watermarks, handling DLQ, performing schema migrations, and providing alerting coverage.

• Conduct code reviews and mentor mid-level engineers.

• Contribute to deployment infrastructure through Helm charts, ArgoCD applications, GKE configuration, Grafana dashboards, and Prometheus alert rules.

• Shape technical direction, including choices for state backend, schema migrations, and the decomposition or reconstruction of pipelines.

⛳️ Requirements

• 5+ years of professional experience in software/data engineering, with a strong proficiency in Java.

• Hands-on experience with Apache Flink and stateful real-time streaming pipelines in production.

• Extensive knowledge of Apache Kafka, including Kafka Connect, consumer groups, partitions, delivery guarantees, and schema management.

• Practical CDC experience utilizing Debezium or Flink CDC.

• Proven experience in designing and operating Kafka → Flink → database/data warehouse pipelines in a production environment.

• Strong knowledge of PostgreSQL and familiarity with at least one analytical database like ClickHouse or BigQuery.

• Experience with Kubernetes and the deployment/operation of production data workloads.

• Demonstrated ability to troubleshoot and optimize production streaming systems—addressing backpressure, checkpoint failures, state growth, latency, lag, and sink performance.

• In-depth understanding of data consistency, idempotency, schema evolution, and observability.

• Capability to work autonomously and manage a data platform component from design to production operation.

• Familiarity with TimescaleDB, hypertables, and time-series data is a plus.

• Knowledge of PostGIS and optimization of geospatial/spatial queries would be advantageous.

• Expertise in ClickHouse performance tuning, including MergeTree and partitioning strategies, is a plus.

• Strong experience in optimizing BigQuery and data modeling is a plus.

• Familiarity with Flink Kubernetes Operator and tuning Flink autoscaler would be beneficial.

• Experience with ArgoCD, Helm, GKE, Prometheus, and Grafana is a plus.

• Background in designing high-throughput, low-latency real-time systems at scale is a plus.

• Experience with MQTT or IoT/event-driven systems would be relevant.

• Knowledge of Kafka Schema Registry, Avro/Protobuf, and advanced schema evolution is a plus.

• Experience in mentoring engineers and leading technical decisions related to streaming architecture would be advantageous.

• Experience in routing, logistics, delivery, mobility, or location-based platforms would be particularly relevant.

🏝️ Benefits

• 20 days of paid vacation.

• 5 sick days.

• Public holidays.

• Flexible schedule with a high degree of autonomy.

• Opportunity to influence both product and technical decisions.

• Professional growth and learning opportunities.

• Comfortable working environment with a supportive team.