Backend Engineer – Data & Orchestration

This is a fully remote position, open to applicants in Germany.

📋 Description

• Take charge of and manage the company's data platform, ensuring it remains dependable, organized, and progressively straightforward.

• Oversee scrapers and platform integrations amidst unreliable sources, rate limits, anti-bot safeguards, and evolving APIs.

• Manage event-driven pipelines comprehensively, from orchestration to databases and data warehouses.

• Develop monitoring and alerting systems for services and integrations.

• Introduce checks for data reliability, including freshness, volume, schema, and value assessments.

• Contribute to production backend services and APIs, particularly involving JavaScript backends.

• Oversee CI/CD processes, infrastructure as code, and cloud infrastructure for data systems.

• Handle access controls and secrets management, including access for AI tools.

• Streamline and document the data system while eliminating outdated components.

• Clarify data capabilities and limitations to colleagues without a technical background.

• Mentor junior engineers as the team expands.

⛳️ Requirements

• 3 to 10 years of practical engineering experience.

• Proven experience managing real production systems in small or rapidly growing organizations.

• Proficient in advanced Python, including asynchronous code and long-running data tasks.

• Backend experience with production services and APIs, including foundational JavaScript or TypeScript backends.

• Familiar with operating unreliable external sources in production, such as scrapers, platform APIs, or unofficial endpoints.

• Understanding of proxies, rate limits, and anti-bot strategies.

• Experience in designing and managing event-driven, fault-tolerant pipelines that process millions of records daily.

• Familiarity with Airflow, Dagster, or task queues.

• Experience with databases like Postgres, ClickHouse, MongoDB, or Redshift.

• Awareness of database and infrastructure operating costs.

• Experience executing checks for freshness, volume, schema, and value.

• Proficient with Datadog or a similar monitoring tool.

• Comfortable with Docker, Terraform, CI/CD, and cloud technologies.

• Ideally, experience with access reviews, role-based access controls, and secrets management.

• End-to-end ownership of a complex data setup in a smaller organization.

• Capable of documenting and maintaining systems effectively.

• Ability to articulate trade-offs to non-technical colleagues.

• Experience in mentoring junior engineers or leading a small team.

• Strong engineering principles and sound technical judgment.

• Ability to utilize AI coding tools to expedite development while maintaining engineering integrity.

• Capability to excel in a fast-paced, performance-driven environment.

🏝️ Benefits

• Equity in the company.

• Flexible remote work, granting full control over your lifestyle.

• Global offsites and regular meetings worldwide.

• Competitive salary (amount not specified).