Manager, Product Analytics, Experimentation

This is a fully remote position, open to applicants in California, +4 more states.

📋 Description

• Oversee and expand the product experimentation initiative, encompassing processes, governance, measurement standards, and team integration.

• Act as the practical subject-matter expert for Optimizely, which includes experiment configuration, audience targeting, quality assurance, instrumentation validation, and test oversight.

• Collaborate with cross-functional teams to pinpoint testing opportunities and design statistically robust A/B, multivariate, and other experimental frameworks.

• Analyze experiment outcomes and convert insights into recommendations that influence product strategy and roadmap choices.

• Work alongside stakeholders to define and assess key digital metrics linked to fan and business objectives.

• Leverage data analysis to uncover trends, patterns, and opportunities that enhance the overall fan experience.

• Develop analytical tools, methodologies, and self-service resources to support data-driven decision-making.

• Convert business inquiries into analytical and experimentation solutions.

• Foster relationships within the North America Insights and Analytics organization.

• Collaborate with Research to merge behavioral, experimental, and qualitative insights for a deeper understanding of fan requirements.

⛳️ Requirements

• Proven experience in leading and implementing a product experimentation program, covering hypothesis formulation, test design, execution, analysis, and recommendations.

• Practical experience with Optimizely or a similar experimentation platform, including setup, quality assurance, launch, and monitoring of experiments.

• Advanced knowledge of statistics, including hypothesis testing, power analysis, sample-size planning, confidence intervals, sequential testing, multiple-comparison adjustments, and variance reduction techniques.

• Demonstrated ability to identify and mitigate common experimentation challenges, such as sample ratio mismatch, peeking, novelty effects, and interactions between concurrent tests.

• Capability to design and execute complex product experiments, including A/B and multivariate tests.

• High proficiency with AI tools and large language models for analytical tasks, with sound judgment in verifying the accuracy of AI outputs.

• Familiarity with digital analytics tools such as Google Analytics and Contentsquare.

• Advanced SQL skills, with practical experience in Databricks and Google BigQuery.

• Exceptional communication and presentation abilities, with the skill to translate intricate analytical and experimental results into clear, engaging narratives for non-technical audiences.

🏝️ Benefits

• Medical, vision, dental, and mental health benefits for you and your family.

• Health care concierge services.

• Flexible Spending Account (FSA) or Health Savings Account (HSA).

• Complimentary concert tickets.

• Generous paid time off, including paid holidays, sick leave, and personal days.

• 401(k) plan with company matching.

• Stock reimbursement program.

• New parent programs, including caregiver leave.

• Support for fertility, adoption, foster care, or surrogacy.

• Career and skill development programs through the School of Live.

• Tuition reimbursement.

• Student loan repayment assistance.

• Volunteer time off.

• Crowdfunding match.