Big Data Infrastructure Engineer

This is a fully remote position, open to applicants in Kazakhstan, +3 more countries.

📋 Description

• Take charge of the big data infrastructure for the international platform and act as the technical lead.

• Deploy, upgrade, and manage core platform services, which include HDFS, S3-compatible object storage (Apache Ozone), Iceberg, and more.

• Assist in conducting an infrastructure pilot that transforms a limited number of very large datasets into thousands of smaller tables.

• Implement compute and storage quotas for various business units.

• Transition from the default Iceberg catalog to a more advanced catalog, such as Lakekeeper or Gravitino.

• Redesign monitoring and alerting systems to ensure alerts are actionable.

• Work closely with an engineer managing the same stack in a parallel environment; exchange strategies and solutions while providing mutual support.

• Participate in architectural decisions and actively replace outdated tools with the latest platform versions.

⛳️ Requirements

• Over 5 years of experience in designing, deploying, and operating big data platforms.

• Practical experience with HDFS/Hadoop, including deployment, version upgrades, and troubleshooting.

• Hands-on experience in deploying and managing S3-compatible object storage; Apache Ozone is preferred, but MinIO or similar solutions are also acceptable.

• Familiarity with Apache Iceberg.

• Experience in establishing monitoring and alerting systems for data infrastructure.

• Proficiency in English at a B2 level.

• Capability to work within the GMT+3 time zone (up to a 3-hour difference).

• Nice to have: Experience with Trino, Kafka, or Flink.

• Nice to have: Familiarity with Iceberg REST catalogs (Lakekeeper, Gravitino, Polaris, or similar).

• Nice to have: Experience in setting resource quotas and cost allocation in a multi-tenant data platform.

• Nice to have: Practical use of AI tools, such as Copilot or agents, to enhance productivity.

🏝️ Benefits

• Flexible working hours and the opportunity to work remotely.

• An ambitious and supportive team that is passionate about their work, values one another, and grows together.

• A structured onboarding process, a personalized development plan, and regular internal lectures and demos from the team.

• Internal programs for adaptation and training, as well as the development of soft skills and leadership abilities.

• Partial reimbursement for participation in external training and conferences.

• Corporate English school offering group and individual lessons, as well as speaking clubs with colleagues worldwide.

• Corporate discounts on hotels and travel services.

• MyTime Day Off - an additional non-working day with full compensation.