Manager, Product Analytics, Experimentation

This is a fully remote position, open to applicants in California, +4 more states.

📋 Description

• Take the lead in developing and expanding the product experimentation program, encompassing processes, governance, measurement standards, and team onboarding.

• Act as the hands-on expert for Optimizely, managing experiment setup, audience targeting, quality assurance, instrumentation validation, and monitoring of tests.

• Collaborate with cross-functional teams to pinpoint testing opportunities and design statistically robust A/B, multivariate, and additional experiments.

• Analyze experiment results and convert findings into actionable recommendations that influence product strategy and roadmap decisions.

• Establish and assess key digital metrics aligned with fan and business objectives.

• Utilize data analysis to uncover trends, patterns, and opportunities that enhance the overall fan experience.

• Develop analytics tools, methodologies, and self-service resources to facilitate data-driven decision-making.

• Convert business inquiries into analytical and experimentation solutions.

• Cultivate relationships within the North America Insights and Analytics organization.

• Collaborate with Research to integrate behavioral, experimental, and qualitative insights for a more profound understanding of fan needs.

⛳️ Requirements

• Proven experience in leading and executing a product experimentation program from hypothesis formulation and test design through to implementation, analysis, and recommendations.

• Practical experience with Optimizely or a similar experimentation platform, including configuration, quality assurance, launching, and monitoring of experiments.

• Advanced understanding of statistical principles, including hypothesis testing, power analysis, sample-size planning, confidence intervals, sequential testing, multiple-comparison correction, and variance reduction techniques.

• Experience in identifying and resolving experimentation challenges such as sample ratio mismatch, peeking, novelty effects, and interaction effects among concurrent tests.

• Capability to design and conduct complex product experiments, including A/B and multivariate testing.

• Advanced skills with AI tools and large language models for analytical tasks, with the discernment to verify the accuracy of AI-generated outputs.

• Familiarity with Google Analytics and Contentsquare.

• High proficiency in SQL, with hands-on experience in Databricks and Google BigQuery.

• Exceptional communication and presentation skills, enabling the explanation of complex analytical and experimental results to non-technical stakeholders.

🏝️ Benefits

• Comprehensive medical, vision, dental, and mental health benefits for you and your family.

• Health care concierge services.

• Flexible Spending Account (FSA) or Health Savings Account (HSA).

• Complimentary concert tickets.

• Generous paid time off, which includes holidays, sick leave, and personal days.

• 401(k) program with company matching.

• Stock reimbursement program.

• New parent programs, including caregiver leave.

• Support for fertility, adoption, foster care, or surrogacy.

• Career and skill development opportunities through the School of Live.

• Tuition reimbursement program.

• Student loan repayment assistance.

• Volunteer time off.

• Crowdfunding matching program.