ML / AI Data Engineer – Contract

This is a fully remote position, open to applicants in India.

📋 Description

• Create, implement, and scale extensive ML and data processing pipelines utilizing cloud infrastructure.

• Develop systems to ingest, process, and deliver over 250,000 hours of multimodal data, including video, audio, and metadata.

• Design and enhance GPU-based computing environments (e.g., NVIDIA Tesla clusters) for both distributed training and inference tasks.

• Construct high-throughput backend systems for video data ingestion from both desktop and mobile platforms.

• Execute distributed processing workflows, focusing on job scheduling, fault tolerance, and resource management.

• Design and establish human-in-the-loop and automated annotation systems to maintain data quality and scalability.

• Convert ML and multimodal research into scalable, production-ready cloud architectures.

• Enhance pipelines for optimal performance, reliability, and cost-effectiveness across computing, storage, and networking layers.

• Collaborate with ML, data, and engineering teams to create comprehensive data workflows.

⛳️ Requirements

• A minimum of 5 years of experience in data engineering, ML pipelines, or distributed systems.

• Significant experience in building scalable data pipelines for large datasets, preferably with video/audio.

• Practical experience with cloud platforms such as AWS, Azure, or GCP.

• Familiarity with GPU-based environments and distributed computing.

• Proficient programming skills in Python, Scala, or similar languages.

• Experience with data processing frameworks like Spark, Ray, Kafka, Airflow, or similar tools.

• Understanding of ML workflows, training pipelines, and inference systems.

• Proven experience in designing fault-tolerant and high-availability systems.

• Strong knowledge of data storage solutions, including data lakes, object storage, and distributed file systems.

• Capability to manage high-throughput, large-scale data ingestion and processing.

🏝️ Benefits

• Equal Opportunity Employer.

• Dedicated to cultivating a diverse and inclusive workplace.

• Encourages applicants from all backgrounds and experiences.