Senior AI Product Engineer

This is a fully remote position, open to applicants in Europe.

📋 Description

• Take full ownership of intricate improvements to Oyster AI, managing the process from problem identification and solution design to evaluation, implementation, and learning during production.

• Create dependable agents and tools featuring well-defined contracts, robust error management, limited execution, and suitable human oversight.

• Enhance knowledge accessibility through strategies involving retrieval, embedding, indexing, and ensuring data freshness while adhering to data-access limitations.

• Develop reliable evaluation loops utilizing representative datasets, experiments, regression assessments, and automated judges aligned with human judgment.

• Identify and resolve production issues across prompts, retrieval methods, tools, models, and infrastructure, followed by verifying fixes with substantiating evidence.

• Make considered trade-offs in model and system design, documenting decisions and criteria for rollout.

• Assist product teams with technical guidance, code reviews, collaborative pairing, and mentoring.

• Advance AI capabilities for Oyster, a worldwide employment platform that supports companies in hiring, compensating, and caring for talent globally.

⛳️ Requirements

• Possess senior-level engineering acumen and a proven record of independently leading complex production projects from design to operation and enhancement.

• Hands-on experience in deploying LLM-powered product features to production, taking personal responsibility for implementation and delivery choices.

• Expertise in retrieval/RAG and indexing, evaluation datasets, experiments, LLM-as-judge, or agent and tool interactions.

• Ability to differentiate between model, retrieval, tooling, and application failures while validating improvements with evidence.

• Strong foundational skills in coding, debugging, testing, API integration, and system architecture.

• Openness to working with Ruby/Rails and React.

• Sound judgment regarding sensitive data, access controls, model constraints, and operational risks.

• Effective communication of trade-offs, proficient asynchronous collaboration, and practical mentoring abilities.

• Regular overlap in working hours with colleagues based in the Americas and Europe, roughly UTC-4 to UTC+1.

• Authorization to work in your place of residence.

• Preferred qualifications: experience in production Ruby on Rails.

• Preferred qualifications: full-stack experience in conversational or assistant technologies.

• Preferred qualifications: experience with Slack or in-app assistant integrations.

• Preferred qualifications: familiarity with managing retrieval pipelines, agent tools, evaluation infrastructure, model routing, and cost controls at a production scale.

• Preferred qualifications: experience in assisting multiple teams in adopting reusable AI patterns.

🏝️ Benefits

• Flexibility to work from anywhere, contingent on timely performance, team support, and authorization to work in your location.

• Enjoy 40 days off annually, including holidays and vacation, or more as mandated by local regulations.

• Access to Plumm for mental well-being services.

• Monthly well-being stipend through the ThanksBen wallet.

• A minimum of three months’ paid parental leave for all new parents.

• Job security for up to 12 months or as required by local laws.

• Work-from-home stipend for laptop and home office essentials.

• Fully remote and asynchronous-friendly work culture.

• Opportunity to engage in real AI products for actual users.

• Ownership of challenging AI engineering tasks.

• Mentorship and the potential for a multiplier effect across product teams.

• Stack-agnostic hiring with opportunities for growth into Rails and React.