Platform Data Engineer

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Design, develop, and maintain **schemas and data models**.

• Enhance table structure, partitioning, indexing, and compression for high-volume datasets.

• Guarantee rapid and efficient querying for logs, requests, metrics, and performance traces.

• Manage ingestion pipelines that handle billions of records.

• Construct resilient pipelines for:

• - API logs

• - Model inference logs

• - Error events

• - Usage & integration events

• - GPU & system metrics

• Execute ETL/ELT workflows to convert raw data into structures ready for analytics.

• Ensure the quality, reliability, and real-time availability of data sources.

• Develop tools to facilitate large-scale **log analysis**.

• Allow in-depth investigation into latency, throughput, errors, and bottlenecks.

• Provide the foundational raw data for end-to-end inference-time monitoring.

• Assist in debugging production issues utilizing logs and traces.

• Collaborate closely with DevOps, ML, and backend engineering teams.

• Integrate pipelines with monitoring tools (Prometheus, Grafana, Datadog, OpenTelemetry).

• Automate ingestion and cleanup processes.

• Create internal libraries or utilities to aid monitoring and debugging workflows.

• Offer clean data interfaces for the Data Expert (dashboards, monitoring, analytics).

• Assist engineering teams by providing the appropriate logs and metrics.

• Participate in debugging, RCA (root cause analysis), and performance optimization efforts.

⛳️ Requirements

• Extensive experience as a **Data Engineer** or in a comparable role within a production setting.

• Strong grasp of **data pipelines**, the difference between streaming and batch processing, and data modeling.

• Familiarity with **analytical databases** (ClickHouse experience is a plus, but not essential).

• Proficient in analyzing **logs, metrics, and platform data** to interpret system behavior.

• Knowledge of **event-driven systems, monitoring, and observability concepts**.

• Pragmatic mindset: you prioritize functionality, reliability, and performance over theoretical concepts.

• Comfortable collaborating across various functions with backend, infrastructure, and data profiles.

• Experience in a startup or scale-up environment is advantageous.

• Nice to Have:

• Experience with high-throughput or real-time systems.

• Exposure to cost monitoring, performance analytics, or platform observability.

• Background in AI, ML platforms, or data-intensive products.

🏝️ Benefits

• Generous paid time off – vacation, sick days, and public holidays.

• Meaningful stock options – share in the success you help create.

• Remote-first environment – work from home wherever we can hire you.

• Flexible hours – manage your schedule outside of core collaboration periods.

• Family leave – paid maternity, paternity, and caregiver time off.

• Company retreats – biannual gatherings in inspiring locations.