Lead Data Mesh Engineer

This is a fully remote position, open to applicants in Spain, +2 more countries.

📋 Description

• Design, provision, and sustain multi-AZ Amazon Managed Streaming for Apache Kafka (AWS MSK) clusters and Apache NiFi environments on Amazon EKS (Graviton ARM64) utilizing Terraform and declarative GitOps methodologies.

• Integrate vendor-agnostic Schema Registries (AWS Glue / Apicurio) to uphold Apache Avro and AsyncAPI data agreements prior to payloads entering production event topics.

• Develop high-performance partitioning strategies, dynamic storage auto-scaling (gp3/io2), and enforce local payload compression (Snappy/Zstd) as mandatory.

• Construct fault-tolerant Dead Letter Queue (DLQ) automated replay loops.

• Ensure that streaming payloads include lightweight, out-of-band tracing context throughout service transitions.

• Take ownership of architectural execution and hands-on delivery of high-throughput, asynchronous event-streaming infrastructure and hybrid data pipelines.

• Architect the Data-in-Motion backbone across multiple cloud environments while ensuring the core on-premises data lakehouse is insulated.

⛳️ Requirements

• BSc/MSc in a relevant field or equivalent professional experience.

• Over 6 years of extensive, hands-on experience in architecting and managing Apache Kafka, preferably AWS MSK, and Apache NiFi within containerized Kubernetes settings (EKS).

• Advanced expertise in Terraform and GitOps deployment tools (ArgoCD or Flux) functioning under a strict pull-based approach.

• Proficient in Apache Avro, AsyncAPI, and schema registry management for Master Data Management (MDM) compliance.

• Proven experience in linking cloud event meshes to enterprise data lakehouses or large-scale data monoliths (Cloudera CDP, Hadoop, Hive).

• Solid understanding of out-of-band telemetry emissions, OpenTelemetry standards, and W3C distributed tracing context propagation.

• Familiarity with Policy-as-Code tools (Open Policy Agent / Rego).

🏝️ Benefits

• Generous PTO Policy.

• Unplugged Days to enhance work-life balance.

• Flexible Work From Home Policy.

• Mental & Physical Wellness programs.

• Phone and Internet Reimbursement program.

• Access to Continued Career Development.

• Comprehensive Benefits and Competitive Packages.

• Paid Volunteer Time.

• Employee Resource Groups.