Senior DevOps Engineer – Data Platform

This is a fully remote position, open to applicants in Germany.

📋 Description

• Manage and take ownership of the end-to-end self-hosted Apache Kafka infrastructure and Schema Registry.

• Oversee Kafka ACLs, client quotas, certificates, upgrades, and disaster recovery processes.

• Operate change data capture pipelines based on Debezium, which includes managing connectors, snapshots, offsets, replica lag, and BigQuery sink loaders.

• Execute Kubernetes workloads, utilizing the Navarch workflow orchestrator to run approximately 7,500 SQL and Python tasks each day.

• Take charge of Kubernetes resource management, including autoscaling and debugging tasks.

• Automate access management and implement infrastructure as code across self-hosted GitLab CI/CD pipelines.

• Substitute manual tickets with code-driven access policies for improved efficiency.

• Manage observability and infrastructure costs through tools such as Datadog, Slack-native alerting, and cost tracking for Kubernetes, Kafka retention, and log volumes.

• Engage in triage duties one week out of three, addressing ingestion and access requests, schema alerts, and providing analyst support.

• Assist in automating triage processes using AI-assisted review methods.

• Serve as a senior sparring partner to two senior data engineers, directly reporting to the Head of Data & Analytics.

• Lead design reviews and initiatives spanning multiple quarters.

• Potentially advance into BigQuery architecture roles focusing on access frameworks, partitioning, slot strategies, pipelines, and models.

• Optionally join the 24/7 on-call rotation.

⛳️ Requirements

• Several years of experience in managing production infrastructure with personal accountability, including participation in on-call rotations.

• Ability to articulate production incidents, propose permanent solutions, and discuss concrete trade-offs supported by quantitative data.

• Extensive hands-on knowledge of Apache Kafka broker operations, including cluster sizing, partition rebalancing, replication, ISR behavior, and performing version upgrades.

• Production experience in cross-system data replication, ideally with Debezium on Kafka Connect or through database replication methods like binlog, GTID, WAL, or agent-based pipelines.

• Strong production background with Kubernetes, focusing on resource management, autoscaling, and debugging complex failure scenarios.

• Proficiency in infrastructure and access management as code across various environments.

• Experience with Terraform and Ansible is preferred; familiarity with Pulumi, Helm, or significant configuration management is also acceptable.

• Adherence to a strict least-privilege IAM approach across multiple environments.

• GCP experience is preferred, or relevant knowledge of AWS/Azure can be considered transferable.

• Proficient in production-grade Python and SQL.

• A cost-conscious mindset regarding query performance, scanning volume, and data pruning.

• Ability to navigate and thrive in a low-process, Kanban-driven environment and self-organize effectively.

• Fluent in English at a C1 level.

• Optional skills or interests in object-oriented design, data modeling/dbt, BigQuery architecture, Airflow, Dagster, Beam, Flink, Spark, n8n, or AI triage bots.

🏝️ Benefits

• Remote work opportunities within Germany or the possibility to work from locations such as Cologne, Darmstadt, Düsseldorf, or Berlin.

• An attractive relocation package for individuals moving to Germany.

• Access to Urban Sports Club and RSG Group Fitness Studios discounts.

• Mental well-being support offerings, including Instahelp for online psychological counseling.

• 30 vacation days per year.

• Opportunity for a sabbatical after a qualifying period.

• Option for Pluxee restaurant vouchers, providing tax-advantaged meal allowances.

• Subsidized Deutschlandticket for travel.

• Monthly employee coupon for Kaufland.de.

• Choice of operating system: MacOS or Ubuntu Linux.

• Access to online language learning programs.

• A variety of in-house training opportunities.

• Automated 360-degree feedback process.

• Coverage for costs related to relevant conferences, training opportunities, and approved team workshops.

• A digital onboarding journey to facilitate smooth integration.

• A buddy program to support new hires.

• Regular team and company events, including all-hands meetings and energizing morning sessions.

• A flat hierarchy, start-up mentality, international team, and agile working environment.