AI Research Scientist – Scientific Evaluation, ML/AI

This is a fully remote position, open to applicants in Switzerland.

📋 Description

• Conduct a critical evaluation of AI-generated ML/AI research papers focusing on their scientific contributions, technical accuracy, methodologies, experimental designs, and conclusions.

• Assess the scientific novelty and importance relative to current state-of-the-art research.

• Examine the rigor and validity of experiments, including baseline comparisons, benchmark selections, evaluation metrics, statistical evidence, and ablation studies.

• Identify any methodological flaws, unsupported assertions, inconsistencies, or misleading outcomes.

• Evaluate reproducibility and technical soundness using available experimental evidence, code, and implementation specifics.

• Review and authenticate relevant academic literature, prior work, citations, and claims of novelty.

• Determine whether AI-generated research aligns with the standards of prestigious ML/AI conferences such as NeurIPS, ICML, and ICLR.

• Deliver structured, evidence-based evaluations that address strengths, weaknesses, critical technical issues, and overall scientific value.

• Utilize detailed evaluation rubrics consistently, accompanied by clear, well-reasoned justifications.

• Provide expert human judgment on the scientific validity, novelty, and comparability of the research to leading work presented at ML/AI conferences.

⛳️ Requirements

• A PhD in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a closely related technical discipline, or equivalent demonstrated research proficiency.

• A minimum of 3 peer-reviewed research publications in NeurIPS, ICML, and/or ICLR (this is a firm requirement), ideally including first-author or substantial contributions.

• Significant hands-on research experience in ML/AI, encompassing the development of novel methodologies, experimental design, result analysis, and publication of original scientific work.

• A solid understanding of scientific methodology and experimental rigor, including appropriate baselines, statistical validity, benchmark development, ablation studies, and reproducibility.

• Capability to independently evaluate the novelty, significance, and technical soundness of advanced ML/AI research.

• Experience in reviewing research papers for top-tier ML/AI conferences or journals, preferably NeurIPS, ICML, or ICLR — highly preferred.

• Familiarity with cutting-edge ML/AI research and recognized benchmarks.

• Proficiency in verifying scientific claims, involving the review of technical arguments, experimental evidence, citations, and relevant code implementations.

• Strong analytical and scientific writing capabilities.

• Ability to apply detailed evaluation criteria and scoring rubrics while exercising independent expert judgment.

• Exceptional attention to detail.

• A CV along with a brief note detailing your research background, areas of expertise, and publication history.

• Links to at least 3 peer-reviewed publications in NeurIPS, ICML, and/or ICLR.

• Relevant experience in academic peer review, if applicable.

🏝️ Benefits

• Fully remote and flexible — work from any location.

• Part-time contractor position with adaptable hours.

• Engage in the evaluation of advanced autonomous AI systems capable of conducting comprehensive scientific research.

• Utilize your research expertise to contribute to the assessment and enhancement of the scientific capabilities of frontier AI agents.