Head of Experimentation

This is a fully remote position, open to applicants in United States.

📋 Description

• Take charge of the Experimentation pillar, overseeing product strategy, roadmap execution, commercial results, and investments in experimentation, warehouse-native analysis, and scalable infrastructure.

• Collaborate with leaders in AI product, observability, and core feature management to establish experimentation as the measurement layer within the AI software development lifecycle.

• Create a seamless process from offline evaluations to production experiments, including automatic promotions and rollbacks, along with self-improving feedback loops for agents.

• Attract sophisticated, high-maturity data science clients by defining and delivering statistical, warehouse, and experiment-workflow capabilities.

• Broaden warehouse-native coverage across significant data warehouses and query layers.

• Provide analysis-only mode, variance reduction, ratio and percentile metrics, exposure validation, and arbitrary-window analysis.

• Operate a high-performing product function, oversee quarterly roadmap commitments, enhance AI-assisted engineering productivity, and recruit to fill gaps.

• Represent the product to external stakeholders, including data scientists, PMs, experimenters, analysts, and partners; translate strategy into actionable insights for the field and prepare sales teams for competitive assessments.

• Achieve improved experimentation win rates, secure strategic reference customers, foster active customer and ARR growth, increase experimentation attach rates, and boost engineering throughput and AI-assisted development adoption.

⛳️ Requirements

• Senior product leader at the GM, VP, or equivalent level with a proven history of managing a product line that competes on statistical accuracy and data infrastructure.

• Extensive operational fluency in experimentation methodologies, including causal inference, variance reduction, ratio metrics, sequential testing, exposure design, multi-armed bandits, and composite/multi-objective metrics.

• Experience conducting experimentation at scale with production data warehouses and non-deterministic systems, where output variance influences sample size and significance determinations.

• Established credibility with data science leaders and experimentation experts in sophisticated organizations, with the capability to recruit them effectively.

• Experience leading a team that includes engineering, design, and data science professionals.

• Ability to develop multi-quarter roadmaps, advocate for investment allocations, and communicate results to executive teams and boards.

• Strong, clear communication skills and quick decision-making capabilities in the face of incomplete information.

• A well-informed perspective on experimentation in an AI-native environment, particularly regarding how agents and autonomous systems utilize experimentation infrastructure.

• Preferred: Experience building or scaling experimentation as foundational infrastructure.

• Preferred: Successfully won competitive evaluations where sophisticated data science organizations were key decision-makers.

• Preferred: Released warehouse-native data products and possesses knowledge of customer data infrastructure operations.

🏝️ Benefits

• 20% Bonus

• Restricted Stock Units (RSUs)

• Health insurance

• Vision insurance

• Dental insurance

• Mental health benefits