Software Engineer, AI

📋 Description

• Participate in AI evaluation pipelines, encompassing offline evaluations, production tracing, and feedback systems;

• Develop and uphold performance metrics (e.g., response quality, task success, reliability) utilizing established frameworks;

• Assist in the creation and maintenance of evaluation datasets and test cases to detect regressions;

• Analyze outcomes and suggest incremental enhancements to model and agent quality;

• Contribute to components of AI systems such as RAG pipelines, retrieval systems, and multi-step workflows within current architectures;

• Write clean and maintainable Python code that integrates with LLM providers and internal services;

• Support enhancements to system reliability, observability, and performance in production;

• Deliver well-defined projects with guidance from more experienced engineers;

• Break down tasks, maintain steady progress, and proactively seek assistance when needed to overcome obstacles;

• Foster team excellence through code reviews, documentation, and knowledge sharing;

• Collaborate with cross-functional partners to launch user-facing features.

⛳️ Requirements

• 2–5 years of professional experience in software engineering;

• Experience in contributing to production systems as a team member;

• Familiarity with AI/ML systems and a strong interest in LLM-powered products;

• Experience in debugging systems, handling data, and iterating on performance;

• Proficiency in Python or a similar programming language;

• Strong grasp of LLM concepts (prompting, RAG, evaluation);

• Knowledge of backend systems, APIs, and cloud environments (e.g., AWS, GCP);

• Experience with logging, monitoring, or debugging tools;

• Willingness to learn tools such as LangGraph, vector databases, and evaluation platforms.

🏝️ Benefits

• Medical insurance;

• Dental insurance;

• Life, AD&D, and Disability Insurance;

• Natural Disaster Support Program;

• Wellness Apps;

• Paid Parental Leave;

• Paid Time off inclusive of holidays and sick time;

• Working Remotely Stipend;

• One-time WFH Office Set-Up Stipend;

• Retirement Plan;

• Financial Planning;

• Learning & Development Budget.