
AI Voice Evaluation Specialist
Posted 3 days ago

Posted 3 days ago
This is a fully remote position, open to applicants in Canada.
• Assess a designated scenario and engage in a natural dialogue using two distinct AI models.
• Execute similar conversations with Model A and Model B, ensuring the scenario, conversational style, and overall engagement remain as uniform as possible.
• Aim to maintain a comparable number of conversational exchanges with each model to facilitate an equitable assessment.
• Record your voice throughout each interaction, paying close attention to the quality and behavior of the models' responses.
• Evaluate each model based on five specified evaluation dimensions.
• Identify and classify relevant error patterns that may emerge in the audio or dialogue.
• Analyze the performance of Model A and Model B according to the evaluation criteria and your observations.
• Choose the model that exhibited superior overall performance based on the established evaluation framework.
• Prepare a comprehensive rationale detailing the final choice, citing specific examples and observations from both dialogues.
• Consistently apply evaluation guidelines across various scenarios and model interactions.
• Bachelor’s degree.
• Exceptional attention to detail and an ability to discern subtle variations in conversational quality.
• Outstanding listening and comprehension capabilities.
• Strong writing skills, particularly in articulating observations clearly and objectively.
• Capability to consistently adhere to detailed evaluation guidelines.
• Comfort in speaking naturally and role-playing various conversational situations.
• Ability to objectively compare two interactions without letting personal biases affect the evaluation.
• Strong critical-thinking and analytical abilities.
• Reliability and consistency in completing structured evaluation assignments.
• Familiarity with AI assistants, voice-based AI, or conversational systems.
• W2 full-time contract employment.
• 40 hours per week.
CVS Health Careers
Sourcegraph
MCA Connect
Lightly
Get handpicked remote jobs straight to your inbox weekly.