Product Lead, AI Agent Evaluation – Research Products

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of 0-to-1 product launches for AI agent and evaluation products, encompassing benchmarks, leaderboards, datasets, developer challenges, research demonstrations, and open-source releases.

• Convert research agendas into actionable product specifications, user journeys, launch strategies, success metrics, and prioritized roadmaps.

• Collaborate with researchers to transform technical work into practical artifacts: documentation, examples, benchmarks, evaluation rubrics, evaluation harnesses, blog posts, landing pages, and demos.

• Lead developer-oriented initiatives such as competitions, cohorts, hackathons, public challenges, and partner-supported benchmark launches.

• Define the product experience surrounding agent improvement cycles: task definition, requirement analysis, skill discovery, adaptation, evaluation, and performance enhancement.

• Coordinate efforts among engineering, design, research, community, marketing, and business development to ensure releases are executed with quality and momentum.

• Identify target user groups, including AI developers, researchers, enterprise teams, open-source creators, and technical partners.

• Gather feedback from users and partners, transforming it into product enhancements, research inquiries, and subsequent releases.

• Craft launch narratives that render technical work comprehensible and engaging without oversimplifying the underlying research.

⛳️ Requirements

• Minimum of 4 years of experience in product management, technical product roles, founder positions, developer platforms, AI tooling, or research-product roles.

• Solid understanding of contemporary AI agents, evaluations, benchmarks, LLM workflows, and open-source/developer ecosystems.

• Demonstrated ability to deliver ambiguous technical products from conception to launch.

• Exceptional writing and communication skills; capable of translating complex research into clear product messaging.

• High level of agency and comfort working across research, engineering, go-to-market strategies, community engagement, and external partnerships.

• Strong product sensibility: recognizes when a research artifact is not yet ready to be a product and understands what is required for it to evolve.

• Proficient in reading academic papers, engaging in discussions about evaluation design, and collaborating with technical users.

• Previous startup experience is advantageous.

• Bonus: familiarity with AI/ML infrastructure, LLM evaluations, Python, open-source launches, developer communities, hackathons, or benchmark design.

🏝️ Benefits

• Competitive salary and equity options.

• Flexible working hours with remote work opportunities.

• Professional development resources and opportunities for growth.

• Collaborative and innovative work environment.