Senior Finance, AI Evaluation Specialist

This is a fully remote position, open to applicants in New York.

📋 Description

• Provide guidance to research and engineering teams regarding financial analysis, valuation, investment strategies, capital allocation, and corporate finance.

• Recognize and address gaps in understanding related to financial models, transactions, market dynamics, forecasting, and strategic decision-making.

• Utilize practical financial judgment in complex situations involving companies, investors, transactions, and commercial objectives.

• Ensure that finance-related tasks accurately represent sound professional decisions and conform to established industry standards.

• Develop challenging, domain-specific tasks that are rooted in real-world financial practices.

• Compose precise and well-reasoned solutions that address scenarios in investment banking, asset management, corporate finance, and financial advisory.

• Create problems that necessitate quantitative analysis, commercial judgment, interpretation of financial data, and assessment of trade-offs.

• Guarantee that tasks are clear, internally coherent, and appropriate for structured evaluation.

• Review AI-generated financial responses against predefined rubrics and scoring standards.

• Evaluate the accuracy, financial judgment, reasoning quality, relevance, and practical applicability of responses.

• Compare different outputs to identify which response offers superior financial analysis.

• Detect calculation inaccuracies, unsupported assumptions, flawed financial reasoning, and incomplete conclusions.

• Provide clear written feedback to facilitate enhancements in model performance.

• Develop and refine evaluation criteria for financial analysis and decision-making tasks.

• Establish scoring criteria that encompass technical accuracy, professional judgment, reasoning, and quality of conclusions.

• Engage in calibration activities with fellow finance specialists.

• Collaborate with various teams to ensure consistency and precision throughout the training data.

• Complete onboarding, specialty calibration, task development, and ongoing quality assessments.

⛳️ Requirements

• Candidates must be located in the United States.

• A minimum of 8 years of dedicated professional experience in finance is required.

• Senior-level expertise in investment banking, asset management, corporate finance, financial advisory, or a related field is essential.

• Experience within a recognized financial institution, investment firm, professional services organization, or similar entity is necessary.

• Prior hands-on experience in evaluating LLM or AI-generated outputs using structured rubrics or scoring criteria is mandatory.

• A demonstrable career trajectory leading to vice president, director, senior manager, or equivalent leadership roles is required.

• Strong financial judgment and the capacity to evaluate complex analytical and commercial decisions are essential.

• Excellent written and verbal communication skills are vital.

• Candidates must be reliably available for at least 35 hours per week during weekdays.

• Previous experience in assessing AI or LLM outputs against structured rubrics is required.

• Applicants need to clearly outline relevant AI evaluation experience in their application.

• A degree in finance, economics, accounting, business administration, mathematics, or a related discipline is highly relevant.

• Graduate-level education in finance, economics, business, or management may be advantageous.

• Equivalent senior professional experience in financial analysis, transactions, or investment management may also be taken into consideration.

• Professional certifications such as CFA, CPA, CA, or similar qualifications may be beneficial.

• Experience in valuation, financial modeling, mergers and acquisitions, capital markets, or portfolio analysis is important.

• A background in transaction advisory, private equity, equity research, strategic finance, or investment management is preferred.

• Experience in creating structured scoring rubrics, evaluation guidelines, or quality assurance frameworks is valuable.

• Familiarity with model training, human-feedback workflows, annotation, or AI quality assurance is desirable.

• A strong understanding of financial statements, forecasting, risk assessment, capital structure, and investment analysis is required.

• Experience in reviewing financial models, investment proposals, transaction materials, or strategic finance documents is necessary.

• Previous collaboration with product, analytics, research, engineering, or data science teams is a plus.

• Immediate availability is preferred.

🏝️ Benefits

• Full-time W-2 contingent employment arrangement.

• Fully remote work environment.

• Competitive hourly compensation.

• At least 35 hours of work per week during weekdays.

• Immediate availability is preferred.

• Flexible project scope and duration based on program requirements and performance.