Senior Data Engineer – AI Infrastructure

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership and enhance streaming data pipelines that facilitate live inference and real-time model serving within Kraken's AI infrastructure.

• Design and construct feature stores that provide low-latency, high-reliability features to production ML models.

• Implement and sustain streaming systems utilizing RisingWave, Apache Flink, or Kafka Streams, choosing the appropriate tool based on the workload.

• Collaborate with ML engineers and AI infrastructure teams to establish data contracts, feature schemas, and pipeline SLAs.

• Propel pipelines towards real-time capabilities where batch processing currently exists, reducing latency from hours to seconds.

• Maintain data quality, observability, and auditability across all streaming and feature engineering systems.

• Contribute to the tooling for inference pipelines where data engineering and model serving converge.

• Assess emerging streaming and feature store technologies to influence the team’s technical roadmap.

⛳️ Requirements

• Over 5 years of experience in data engineering, with a minimum of 2 years dedicated to streaming systems in a production setting.

• Practical experience with RisingWave, Apache Flink, Kafka Streams, or similar stream processing frameworks.

• Strong grasp of feature store design, including online/offline consistency, point-in-time accuracy, and low-latency serving.

• Proven experience in constructing data pipelines that support production ML models or inference systems.

• Expertise in Python and/or Scala; fluency in SQL is essential.

• Knowledge of data quality frameworks, pipeline observability, and ownership of SLAs.

• Adept at working within a fast-paced and ambiguous environment, integrated within an AI-focused team.

🏝️ Benefits

• Health insurance

• 401(k) matching

• Wellness allowance

• Remote work options