Data Scientist – Responsible AI

This is a fully remote position, open to applicants in United States.

📋 Description

• Conduct statistical analysis and experimentation to assess generative AI, agentic AI, and various AI-enabled applications.

• Create and uphold Responsible AI standards, benchmark datasets, testing suites, and evaluation criteria.

• Assess AI systems for fairness, bias, explainability, transparency, hallucination, safety, reliability, and robustness.

• Utilize evaluation findings to aid in the design, testing, selection, and ongoing evaluation of AI-enabled business applications.

• Set performance baselines and assist in identifying changes in model or application behavior over time.

• Design and implement proofs of concept for innovative Responsible AI evaluation methodologies.

• Evaluate emerging Responsible AI tools, frameworks, and methodologies for their validity, limitations, and applicability to business needs.

• Facilitate the integration of Responsible AI testing within application development, quality assurance, and governance processes.

• Collaborate with engineering, architecture, quality assurance, governance, and business teams to align evaluation methods with use case demands.

• Develop and maintain model cards, evaluation reports, methodology documentation, and associated guidance.

• Present findings, limitations, risks, and recommendations to both technical and nontechnical stakeholders.

• Contribute to Responsible AI training, internal education, applied research, and industry outreach.

• Keep abreast of emerging technologies and evaluation techniques related to generative AI, agentic AI, and Responsible AI.

⛳️ Requirements

• Bachelor's degree in Data Science, Computer Science, Statistics, Mathematics, Psychology, Economics, Operations Research, or another quantitative field.

• 2 to 5 years of experience in data science, statistical analysis, AI evaluation, applied research, or analytics.

• Proficient in Python programming with experience in data analysis and AI evaluation tools.

• Strong SQL skills and experience managing large, complex datasets.

• Background in designing experiments, selecting relevant measures, and interpreting statistical results.

• Experience in evaluating generative AI applications, large language models, agentic systems, or other AI-enabled solutions.

• Familiarity with Responsible AI concepts, including fairness, bias, explainability, transparency, hallucination, safety, reliability, and robustness.

• Experience in developing evaluation datasets, benchmarks, test cases, scorecards, or performance metrics.

• Ability to translate evaluation results into actionable recommendations for business applications and technical teams.

• Knowledge of source control, automated testing, technical documentation, and basic software engineering practices.

• Capability to evaluate technical methodologies, identify limitations, and communicate findings effectively.

• Excellent written and verbal communication skills.

• Ability to work independently and collaborate across technical, governance, and business teams.

🏝️ Benefits

• Medical, dental, vision, and life insurance.

• Retirement savings – 401(k) plan with generous company matching contributions (up to 6%), financial advisory services, potential company discretionary contribution, and a wide range of investment options.

• Tuition reimbursement up to $5,250 per year.

• A business-casual environment that allows for wearing jeans.

• Generous paid time off upon hire – including a paid time off program along with ten paid company holidays and three floating holidays each calendar year.

• Paid volunteer time – 16 hours per calendar year.

• Leave of absence programs – including paid parental leave, paid short- and long-term disability, and Family and Medical Leave (FMLA).

• Business Resource Groups (BRGs) – BRGs promote inclusion and collaboration across our organization and within the communities where we live, work, and engage. BRGs are open to all.