Senior Machine Learning Engineer II, Ads Response Prediction

This is a fully remote position, open to applicants in California, +18 more states.

📋 Description

• Spearhead the research and development of pCTR and conversion prediction models, concentrating on enhancing calibration, minimizing training data biases (such as selection bias, position bias, and optimizer’s curse), and improving model accuracy across Instacart’s advertising platforms.

• Create and execute debiasing strategies like Mixed Negative Sampling (MNS), Inverse Propensity Weighting (IPW), counterfactual risk minimization, and calibration techniques (including Platt scaling and isotonic regression) to tackle systematic prediction biases.

• Contribute to the cutting-edge Multi-Domain Multi-Task (MDMT) model architecture, integrating innovations such as Mixture-of-Experts (MoE), Transformer layers for sequential user behavior, and LoRA adaptors for scalable domain fine-tuning.

• Lead sequence modeling projects including the TIGER generative retrieval system and Semantic ID representation learning, broadening their usage across advertising surfaces like Product Details, Search, and other placements.

• Collaborate with the wider ML community within the company on the journey toward Foundation Models utilizing autoregressive user behavior prediction.

• Define and scope ambiguous modeling challenges from foundational principles. Convert business insights (e.g., overcalibration trends, cold-start underperformance) into clearly articulated ML research pathways with defined evaluation metrics.

• Publish and share results internally. Foster the team’s culture of technical excellence through design reviews, sharing research papers, and conducting experiment retrospectives.

⛳️ Requirements

• PhD/Master’s degree in machine learning, statistics, computer science, information retrieval, or a closely related quantitative discipline.

• Over 6 years of combined academic and industry experience (including PhD research) applying ML to ranking, recommendation, or prediction challenges at scale.

• Profound understanding of CTR/conversion prediction modeling, including experience with architectures such as Deep & Wide, DeepFM, DCN, and multi-task learning approaches.

• Strong grounding in causal inference, counterfactual reasoning, and strategies for mitigating training data biases. Ability to analyze selection bias, position bias, and propensity-based correction techniques.

• Proficiency in Python and deep learning frameworks (including PyTorch, TensorFlow, JAX). Familiarity with data manipulation tools (such as SQL, Spark, Pandas).

• Proven experience in transforming ambiguous problems into well-defined ML research directions and achieving results through thorough experimentation.

• Excellent written and verbal communication skills. Capability to articulate complex modeling decisions to cross-functional stakeholders, including product managers and data scientists.

🏝️ Benefits

• Highly competitive market compensation

• Eligibility for a new hire equity grant

• Annual refresh grants