Senior Staff Data Scientist – Consumer Relevance

This is a fully remote position, open to applicants in Canada.

📋 Description

• Act as the technical expert on relevance metrics and evaluation methodologies within Consumer, establishing benchmarks for assessing the quality of feeds, search results, and recommendations in a multifaceted, community-focused environment.

• Create metrics frameworks and offline evaluation strategies for ranking and recommendation systems, including proxy metrics that consistently predict long-term results such as retention, community health, and user satisfaction.

• Design and evaluate experiments for relevance features, addressing challenges unique to networked platforms, including spillover effects among communities, interference between contributors and consumers, and the long-term impacts of ranking modifications on content supply.

• Discover opportunities where enhanced measurement and analysis can reveal product insights that were previously difficult to quantify or unclear, especially concerning content quality, understanding search intent, and the effectiveness of personalization.

• Collaborate closely with ML engineers and product teams to translate model performance metrics into user-facing impacts.

• Shape the long-term product strategy for Feeds and Search by integrating insights derived from experimentation, observational analysis, and metric evaluations into clear, actionable recommendations for senior leadership.

• Guide and uplift other data scientists throughout the organization regarding relevance evaluation, best practices in experimentation for ranking systems, causal reasoning, and statistical rigor.

• Publish and disseminate methodological advancements internally, and where appropriate, externally to contribute to the wider relevance, recommendation systems, and experimentation community.

⛳️ Requirements

• Ph.D. in Statistics, Computer Science, Information Retrieval, Economics, or a related quantitative discipline with a strong emphasis on recommendation systems, ranking, causal inference, or evaluation methodology; or M.S. with comparable expertise.

• For M.S. holders: 12+ years of industry experience in applied science, data science, or roles focused on relevance/ranking.

• For Ph.D. holders: 8+ years of industry experience in applied science, data science, or relevance/ranking-focused roles.

• In-depth expertise in metrics design and evaluation for ranking and recommendation systems, encompassing offline metrics and counterfactual evaluation.

• Strong comprehension of causal inference and experimental methodologies, including practical experience with challenges pertinent to ranking systems such as novelty effects, position bias, long-run effect estimation, and ecosystem-level impacts.

• Experience in defining and validating quality metrics for content ranking, search, or recommendations at scale.

• Solid theoretical foundation in experimental design, including power analysis, variance reduction techniques, and sequential testing as applied to relevance experiments.

• Expert proficiency in SQL and competent in R and/or Python for statistical analysis.

• Proven track record of influencing product and organizational strategy through data-driven insights related to content quality and user experience.

• Exceptional communication skills, capable of articulating nuanced statistical and ML concepts and trade-offs to both technical and non-technical senior stakeholders.

• Experience mentoring data scientists and enhancing organizational capabilities in relevance evaluation and experimentation.

• Comfortable in innovative and fast-paced settings with a proactive approach to action.

🏝️ Benefits

• Global benefit programs tailored to your lifestyle, encompassing workspace, professional development, and caregiving support.

• Family planning assistance.

• Gender-affirming care.

• Mental health and coaching benefits.

• Comprehensive medical benefits and health care spending account.

• Registered Retirement Savings Plan with matching contributions.

• Income replacement programs.

• Flexible vacation and paid volunteer time off.

• Generous paid parental leave.