AI Voice Evaluation Specialist

This is a fully remote position, open to applicants in United States.

📋 Description

• Analyze a designated scenario and engage in a natural dialogue with two distinct AI models.

• Facilitate similar discussions with Model A and Model B, ensuring the scenario, conversational style, and overall interaction remain as uniform as possible.

• Aim to maintain an equivalent number of conversational turns with both models to facilitate a fair evaluation.

• Record your voice during each session and assess the quality and responsiveness of the models' replies.

• Evaluate each model based on five established assessment dimensions.

• Identify and classify relevant error patterns in the audio or conversation.

• Compare Model A and Model B using the evaluation criteria and your observations.

• Choose the model that exhibits superior overall performance.

• Compose a comprehensive rationale detailing the final choice, supported by specific examples and observations.

• Consistently apply evaluation guidelines across various scenarios and model interactions.

⛳️ Requirements

• A bachelor's degree.

• Strong attention to detail with the ability to discern subtle variations in conversational quality.

• Excellent listening and comprehension abilities.

• Proficient written communication skills, especially in articulating observations clearly and objectively.

• Capability to consistently adhere to detailed evaluation guidelines.

• Comfort in engaging naturally and role-playing various conversational scenarios.

• Ability to impartially compare two interactions without personal biases affecting the evaluation.

• Strong critical-thinking and analytical capabilities.

• Dependability and consistency in performing structured evaluation tasks.

• Familiarity with AI assistants, voice-based AI, or conversational systems.

🏝️ Benefits

• W2 employment.

• Full-time contract for a duration of 3 months.

• 40 hours of work per week.