Senior Marketing Strategy, AI Evaluation Specialist

This is a fully remote position, open to applicants in New York.

📋 Description

• Provide guidance to research and engineering teams on brand strategy, growth marketing, performance marketing, and concepts related to campaign planning.

• Identify discrepancies in model comprehension regarding customer segmentation, positioning, channel strategies, creative development, and measurement.

• Utilize practical marketing insights to navigate complex situations involving brands, target audiences, markets, and commercial goals.

• Ensure that marketing tasks align with realistic business decisions and adhere to professional standards.

• Create challenging, domain-specific tasks that are rooted in authentic marketing practices.

• Compose accurate and well-supported solutions addressing brand, growth, campaign, and performance-marketing scenarios.

• Formulate problems that necessitate strategic judgment, prioritization, data interpretation, and evaluation of trade-offs.

• Guarantee that tasks are clear, internally consistent, and appropriate for structured assessment.

• Evaluate AI-generated marketing responses against established rubrics and scoring criteria.

• Assess the accuracy, strategic judgment, reasoning quality, relevance, and practical applicability of responses.

• Compare various outputs to determine which provides the most robust marketing recommendation.

• Document factual inaccuracies, unsupported assumptions, insufficient strategic logic, and incomplete analyses.

• Offer clear written feedback that facilitates improvements in model performance.

• Develop and enhance evaluation criteria for brand, growth, campaign, and performance-marketing tasks.

• Define scoring criteria for strategic quality, commercial judgment, reasoning, and feasibility of execution.

• Engage in calibration activities alongside other marketing professionals.

• Collaborate with various teams to ensure consistency and accuracy throughout the training data.

⛳️ Requirements

• Senior marketing professional based in the US.

• A minimum of 8 years of dedicated professional experience in the marketing field.

• Senior-level expertise in brand strategy, growth marketing, performance marketing, campaign strategy, or related disciplines.

• Experience with a recognized consumer brand, agency, technology firm, or a similar organization.

• Prior experience in evaluating LLM or AI-generated outputs using structured rubrics or scoring criteria is essential.

• Demonstrated career advancement into senior manager, director, vice president, or similar leadership roles.

• Strong commercial judgment skills.

• Capability to assess complex marketing decisions.

• Exceptional written and verbal communication abilities.

• Reliable availability for a minimum of 35 hours per week during weekdays.

• A degree in marketing, business administration, communications, economics, consumer psychology, or a related field is highly relevant.

• Graduate-level education in marketing, strategy, business, or management may be advantageous.

• Equivalent senior professional experience in brand, growth, or performance marketing may also be taken into account.

• Professional training in analytics, market research, digital marketing, or AI evaluation may be beneficial.

• Applicants should clearly articulate relevant AI evaluation experience in their application.

• Immediate availability is preferred.

🏝️ Benefits

• Full-time remote engagement.

• Competitive hourly compensation.

• W-2 contingent employment arrangement.

• Remote consulting opportunity.

• Immediate availability preferred.

• Work may include onboarding, specialty calibration, task development, and ongoing quality review.