Lead AI Scientist

This is a fully remote position, open to applicants in Texas.

📋 Description

• Create comprehensive, modular AI systems that encompass knowledge and data inputs, retrieval mechanisms, models, agents, tools, deterministic services, human review, evaluation, and integration.

• Decompose intricate problems into fitting combinations of predictive ML, statistical techniques, generative models, retrieval processes, graph reasoning, deterministic software, and human decision-making points.

• Develop functional systems and reference implementations, collaborating with software engineers to incorporate them into Pearson’s assessment authoring platform.

• Design architectures that adapt as models, data, domain knowledge, and evaluation evidence evolve.

• Craft and execute agentic and multi-step workflows utilizing models, tools, retrieval methods, state management, structured outputs, and human decision points.

• Construct knowledge-based systems employing graph, semantic, vector, and hybrid retrieval approaches.

• Collaborate with the AI Data & Knowledge Architect to co-design knowledge and graph architectures.

• Advance AI capabilities for content generation and revision, verification, framework alignment, guidance, and analytic insights.

• Establish and maintain evaluation benches, encompassing expert-adjudicated datasets, challenge sets, regression suites, and protected holdout data.

• Work alongside psychometricians and content specialists to translate domain expectations into quantifiable criteria.

• Investigate failure modes, critical errors, and performance across significant data and task segments.

• Supervise deployed systems for drift and emerging failures, using insights to promote enhancements.

• Create predictive models, classifiers, and ranking methods that outperform generative AI.

• Assess tradeoffs in prompting, retrieval, structured knowledge, model selection, and model adaptation.

• Implement parameter-efficient fine-tuning and other adaptation techniques when validated by evaluation evidence.

• Develop versioned evaluation and training resources from expert assessments and adjudicated examples.

• Design synthetic data methodologies with stringent quality controls.

• Rapidly prototype, empirically test assumptions, and pivot based on evidence.

• Construct precise, transparent, and auditable AI systems that enhance rather than replace human expertise.

• Adhere to responsible AI and data practices, including privacy, security, fairness, and appropriate handling of student and content data.

• Collaborate with technology teams, software engineers, AI Science colleagues, the AI Data & Knowledge Architect, psychometricians, learning scientists, and content development teams.

• Effectively communicate findings, tradeoffs, and limitations to both technical and non-technical audiences.

⛳️ Requirements

• Master's degree in Computer Science, Artificial Intelligence, Machine Learning, Statistics, or a related discipline, or equivalent experience.

• Proven experience in designing and constructing comprehensive AI or machine-learning systems end-to-end, rather than isolated models or prompts.

• Extensive knowledge across predictive ML, statistical learning, and LLM-based, agentic, and retrieval-grounded systems, with discernment regarding appropriate applications for each.

• Demonstrated expertise in evaluating probabilistic systems against ground truth or expert judgment, including evaluation datasets, error analysis, and regression testing.

• Experience with knowledge graphs and knowledge-grounded AI sufficient to participate in architectural decisions.

• Strong proficiency in Python.

• Familiarity with TypeScript or the ability to quickly become productive in a TypeScript codebase.

• A history of innovating beyond established practices, showcasing sound technical judgment and a high degree of autonomy.

• Ability to collaborate closely with engineers and domain experts, translating specialized knowledge into requirements and evaluation criteria.

• Preferred: Experience in developing knowledge and inference layers for education, learning, or assessment systems.

• Preferred: Experience in creating AI or ML systems within evidence-sensitive domains.

• Preferred: Practical experience with parameter-efficient fine-tuning or other model adaptation techniques.

• Preferred: Experience in designing, filtering, and validating synthetic data.

• Preferred: Experience in developing expert-annotated datasets, adjudication protocols, or evaluation platforms.

• Preferred: Experience with graph-grounded agents, hybrid retrieval, or knowledge-intensive AI applications.

• Preferred: Familiarity with ontologies, semantic models, or hybrid neural-symbolic architectures.

• Preferred: Experience in calibration, uncertainty estimation, drift detection, or model monitoring.

• Preferred: Familiarity with TypeScript, AWS, PostgreSQL, graph data stores, Go, Python, and ML lifecycle tools such as MLflow.

• Preferred: Understanding of educational measurement, psychometrics, or learning science.

🏝️ Benefits

• Eligibility for annual incentive programs.

• Comprehensive benefits information provided by Pearson.