Research Engineer

This is a fully remote position, open to applicants in United States, +1 more country.

📋 Description

• Develop evaluation frameworks for AI agents, which include datasets, sandbox environments, and performance metrics on real-world tasks.

• Design experiments that compare models, agent harnesses, tools, and execution environments, while identifying regressions and trade-offs related to performance, reliability, and cost.

• Transform qualitative feedback regarding agent behavior into research questions, structured experiments, and quantifiable outcomes.

• Examine agent failures to ascertain whether problems stem from the model, harness, tooling, or prompting.

• Create strategies to enhance agent performance.

• Build and enhance agent harnesses, internal tools, and integrations, including contributions to Aria and other business agents.

• Foster shared knowledge and reusable skills across agents.

• Collaborate with Infrastructure and Security teams to guarantee reliable and safe production operations.

• Implement engineering enhancements and integrations, subsequently engaging in open-ended research concerning evaluation, agent behavior, and infrastructure.

⛳️ Requirements

• Practical experience in building, evaluating, or refining LLM models or agents, through professional assignments, applied research, open-source contributions, or significant personal projects.

• Proficient in Python with solid software engineering principles.

• Skills in production-quality coding, with an understanding of testing, maintainability, and reliability.

• Background in applied research, machine learning, or data science, which includes designing experiments, managing datasets, and deriving conclusions from results.

• Knowledge of LLM agents that goes beyond prompting, such as tool usage, harnesses, context management, and common failure modes.

• Capability to decompose open-ended problems into testable questions while working independently.

• Experience in creating tools or systems utilized by others and assessing their effectiveness beyond technical benchmarks.

• Comfort in navigating both research and engineering environments, collaborating with various teams, and adapting to changing priorities.

• A genuine interest in AI agents.

• Candidates must be eligible to work within the contiguous United States and Canada.

• An advanced degree or postgraduate research experience in computer science, mathematics, statistics, machine learning, or a related field is advantageous but not mandatory.

🏝️ Benefits

• Remote-first policy — work from any location within the United States.

• Regular in-person company retreats and a cross-country "office visit" perk.

• 100% paid medical, dental, and vision premiums for employees.

• $1,000 stipend for working from home.

• Monthly reimbursements for home internet, phone, and cellular data expenses.

• Unlimited vacation policy.

• Fully paid parental leave of 12 weeks.

• Fertility benefits available.

• Opportunity for incentive compensation.

• Equity stake in the company.