Lead ClickHouse Engineer

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Managing cluster operations at scale — overseeing, upgrading, and developing a multi-AZ ClickHouse environment on Kubernetes, complete with practiced backup and recovery procedures.

• Schema and query engineering — designing tables and sorting keys, implementing partitioning, creating materialised views, and optimizing queries for multi-terabyte datasets.

• Capacity and observability — utilizing a defined capacity model, implementing storage tiering and retention strategies, and employing Grafana for monitoring and alerting to maintain visibility of system health.

• Ingestion and data quality — establishing correctness and completeness checks on the data pipelines feeding the system; ensuring reconciliation processes confirm that no data is lost.

• Workload isolation — ensuring that ingestion, operations, reporting, and ad-hoc analytics do not interfere with each other as read loads increase.

• Team and knowledge leadership — mentoring the engineers involved with the system, setting standards for runbooks and documentation, and ensuring the system is maintainable by the team rather than solely by the author.

• AI-assisted operations — leveraging modern tools (including MCP-based AI access) that empower a small team to effectively manage a large platform.

⛳️ Requirements

• Expertise in columnar/OLAP database engineering — ClickHouse is highly preferred; substantial experience with another columnar or large-scale time-series database (such as BigQuery, Redshift, Druid, kdb+, or similar) is also acceptable.

• Proven experience in making architectural decisions for a production data platform, rather than merely operating within a pre-designed system.

• Proficient in SQL — capable of analyzing execution plans, optimizing queries, and designing schemas for very large datasets.

• Operational/SRE experience — having held production responsibility for a data platform including monitoring, capacity management, incident handling, and recovery processes.

• Familiarity with Linux and Kubernetes — comfortable with the operational foundations supporting the database.

• Proficient in at least one programming language — ideally Python, but also Go, R, or MATLAB are acceptable.

• Engineering discipline — skilled in drafting proposals, strategies, and architectural documents and diagrams, as well as managing change effectively.

• Proficient in Git — demonstrating excellent version-control practices.

• Understanding of distributed systems fundamentals — including replication, consistency, and failure modes.

• Experience in financial services is advantageous but not mandatory — domain knowledge can be acquired; operational instinct is essential.

🏝️ Benefits

• A substantial estate — genuine scale and critical importance: serving as the system of record for a live trading network, rather than a mere reporting tool.

• True ownership — providing architectural and technical leadership for a vital core platform within the business.

• An outstanding team — collaborating with senior engineers who value craftsmanship, engage in open communication, and maintain high standards.

• Sustainable operations — designed for follow-the-sun coverage; on-call duties are shared, contracted, and planned without the need for heroics.

• Competitive package — offering compensation and benefits that are competitive and aligned with your local market.

• Modern tooling — utilizing AI-assisted operations and investigative tools that reduce toil instead of complicating processes.

• Growth opportunities — the estate is expanding to multiple times its current volume; the role will evolve along with it.