Pipeline Engineer

This is a fully remote position, open to applicants in Spain, +4 more countries.

📋 Description

• Design, construct, and manage automated pipelines from start to finish.

• Take ownership of the automated protection pipeline, encompassing threat intelligence through to validated and deployed rules.

• Develop progressive release automation featuring controlled stages along with automated hold or rollback safeguards.

• Establish quality gates utilizing live production signals.

• Create CI validation with ephemeral environments across various software versions and configurations.

• Coordinate LLM subsystems alongside evaluation, budgeting, and expense tracking.

• Develop observability, metrics, dashboards, alerting systems, and health gates.

• Transform fragile multi-stage batch jobs into resumable, idempotent, observable systems with defined state machines and recovery pathways.

• Set and enforce latency budgets and SLOs for each stage.

• Design guardrails that include blast-radius limits, kill switches, and safe-by-default practices.

• Eliminate manual operational tasks and minimize maintenance burdens.

• Write and uphold unit and integration tests for concurrency, partial failures, API inconsistencies, and multi-stage states.

• Investigate and address issues across ClickHouse, GitLab CI, S3/object storage, Prometheus/Grafana, and third-party APIs.

• Collaborate with security analysts and the Server team on architectural decisions.

⛳️ Requirements

• Over 5 years of professional experience in backend, platform, or infrastructure engineering.

• Proven experience in building and managing multi-stage data or automation pipelines.

• Significant expertise in at least one of the following: Python, Go, or Rust.

• Strong systems design judgment.

• Practical experience with workflow orchestration and job scheduling.

• Familiarity with concepts such as idempotency, retries with backoff, exactly-once versus at-least-once delivery, checkpointing and resumability, graceful degradation, backpressure, and effective handling of partial failures.

• Hands-on experience in observability, including the design of metrics.

• Extensive CI/CD experience, preferably with GitLab CI, including dynamic/child pipelines and self-hosted runners.

• Proficiency with Docker and container-based testing environments.

• Experience with object storage solutions like S3/Ceph or equivalent.

• Familiarity with large-scale analytical databases such as ClickHouse or another columnar database.

• Capability to design state machines and long-running processes that can withstand restarts.

• Ability to analyze concurrency across multiple concurrent rollouts.

• Excellent debugging abilities across system, network, and data layers.

• Strong communication skills and the ability to work effectively in a distributed team environment.

• Proficient in both spoken and written English.

🏝️ Benefits

• Emphasis on professional growth and development.

• Engaging and challenging projects.

• Fully remote work with flexible hours, enabling you to plan your day and work from any location globally.

• Paid vacation of 24 days each year.

• 10 public holidays.

• Unlimited sick leave.

• Compensation for private health insurance.

• Reimbursement for co-working expenses.

• Gym and sports reimbursement.

• Budget allocated for education.

• Opportunity to earn a reward for the most innovative idea that the company can patent.