Psychology Research Specialist – AI Benchmarking

This is a fully remote position, open to applicants in New York.

📋 Description

• Create original multiple-choice questions in psychology that assess deep conceptual understanding.

• Design challenging questions across various areas including psychological theory, research methodology, psychometrics, human factors, and applied behavioral science.

• Review existing questions for accuracy, clarity, rigor, completeness, and empirical support.

• Edit questions and answer options while documenting significant changes with justifications.

• Utilize graduate-level theory and empirical research in the design of assessments.

• Assess study designs, experimental evidence, measurement techniques, and research conclusions.

• Develop questions that incorporate behavioral data, competing theoretical perspectives, and methodological strategies.

• Formulate psychometrics and measurement problems relating to reliability, validity, scale construction, and measurement error.

• Generate content related to human factors and engineering psychology, focusing on cognition, decision-making, attention, perception, and human-system interaction.

• Create and evaluate content regarding digital health, technology-mediated behavioral interventions, neuromorphic engineering, and psychedelic-assisted therapy.

• Provide 1–5 academic references for each question as needed.

• Develop structured explanations for solutions and assess the difficulty of questions.

• Contribute to high-quality evaluation materials for advanced AI capabilities.

• Participate in either question-authoring or question-verification workstreams.

⛳️ Requirements

• PhD, PsyD, or current doctoral candidacy in Psychology or a closely related field.

• A Master's degree may be considered for candidates with exceptional expertise in a relevant subdomain.

• Strong command of graduate-level psychological theory, research methodology, and empirical literature.

• Expertise in one or more areas such as Human Factors, Engineering Psychology, Consumer Psychology, Market Psychology, Psychometrics, Digital Health, Psychedelic-Assisted Therapy, or related behavioral-science fields.

• Solid understanding of experimental design, evidence evaluation, and psychological measurement.

• Excellent written English skills and the ability to articulate complex concepts clearly and precisely.

• Academic research publications are highly regarded.

• Clinical licensure is beneficial for relevant areas of specialization.

• Experience in university-level teaching, assessment design, or examination development is highly valued.

• Strong attention to conceptual precision, methodological validity, and the quality of evidence.

• Expected availability of at least 10 hours per week.

• Must work without using any confidential or proprietary information belonging to any employer, client, institution, or other third party.

• H1-B and STEM OPT support is not available for this position.

🏝️ Benefits

• Flexible scheduling based on project requirements.

• Fully remote and asynchronous working environment.

• Part-time independent contractor engagement.

• Project duration may be adjusted based on needs and performance.