Machine Learning Data Engineer, LLM, Generative AI

This is a fully remote position, open to applicants in New York.

📋 Description

• Design, construct, and sustain scalable automated data pipelines for LLM fine-tuning, encompassing SFT, RLHF, and DPO.

• Develop telemetry and sampling systems for the ingestion of real-time production data.

• Assess, analyze, and benchmark the performance, latency, and accuracy of live LLM and AI agents.

• Architect and scale synthetic data generation workflows to enhance training data, address edge cases, and reduce bias.

• Implement automated checks for data quality, deduplication methods, and toxicity filtering.

• Integrate data pipelines with model training infrastructure and facilitate CI/CD for model updates.

• Manage and optimize data flows into vector databases for RAG functionalities.

• Collaborate closely with ML Researchers and DevOps teams.

⛳️ Requirements

• A Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or a related technical discipline.

• Over 5 years of professional experience in Data Engineering, Machine Learning Engineering, or MLOps.

• Expert-level skills in Python and advanced proficiency in SQL.

• Proven hands-on experience with LLM APIs (such as OpenAI, Anthropic), open-source models (like Hugging Face, Llama), and orchestration frameworks (including LangChain, LlamaIndex, or AutoGen).

• Demonstrated capability in utilizing prompt engineering and programmatic approaches for high-quality synthetic data generation and validation.

• Strong experience with workflow orchestration tools.

• Familiarity with scalable cloud infrastructure (GCP) and distributed data processing frameworks.

• Equivalent qualifications in the specified requirements will also be taken into account.

🏝️ Benefits

• Competitive salary.

• Initial stock options grant.

• Annual performance bonus.

• Unlimited PTO.

• Paid parental leave.

• Work model: currently fully remote, with plans for a future hybrid setup.

• Health, dental, and vision insurance plans.

• 401(k) with employer matching for employees based in the US.

• Continuous learning opportunities.

• Empowering prospects for growth in a dynamic entrepreneurial environment.