Voice AI QA Analyst

This is a fully remote position, open to applicants in California.

📋 Description

• Execute structured User Acceptance Testing (UAT) on AI voice agents prior to client launch by making and receiving test calls, exploring edge cases and failure modes, and documenting reproducible defects.

• Evaluate scripted and adversarial scenarios, which include identity verification, payment negotiation, and escalation or handoff triggers.

• Oversee the ongoing performance of live agents across client deployments by utilizing call-level quality assurance and aggregate performance metrics.

• Listen to and assess live agent calls against Veritus's quality assurance scorecard, focusing on accuracy, compliance, and tone.

• Analyze call data to identify trends, recurring failure modes, outliers, and opportunities for performance enhancement.

• Monitor performance metrics over time and investigate any changes or anomalies in agent behavior.

• Collaborate with Customer Success to compile quarterly client performance reports that combine quantitative analysis with insights from call reviews.

• Convert call and performance data into actionable recommendations for clients and internal teams.

• Identify systemic issues, such as recurring failure patterns, scorecard drift, and emerging edge cases, and communicate these to Customer Success and engineering departments.

• Maintain comprehensive documentation of test results, call assessments, analyses, and findings within tracking tools.

⛳️ Requirements

• 2–4 years of experience in QA, quality analysis, operations analysis, or a related field, preferably in a consumer lending contact center or voice-based environment.

• Knowledge of US consumer lending laws and regulations, including FDCPA, TCPA, ECOA, and SCRA.

• Exceptional attention to detail and proficiency in following structured test plans and scorecards.

• Comfort in working with quantitative data and basic statistics to discern trends and evaluate performance.

• Experience with spreadsheets or data analysis tools to assess operational or customer interaction data.

• Ability to translate quantitative and qualitative findings into clear conclusions and actionable recommendations.

• Proficient written English skills for creating defect reports, call notes, and client-facing performance summaries.

• Willingness to give and receive detailed feedback regarding quality issues.

• Experience in testing AI/voicebot or IVR systems is a plus but not mandatory.

• Previous experience in UAT or pre-release testing for software products is a plus but not mandatory.

• Familiarity with analyzing contact-center or call-performance data is a plus but not mandatory.

• Experience in creating client-facing performance reports or collaborating closely with Customer Success teams is a plus but not mandatory.

• Familiarity with SQL, BI tools, or other data analysis tools is a plus but not mandatory.

• A Bachelor's degree is preferred; equivalent professional experience in QA, operations, or analytics is acceptable.

• Must be legally authorized to work in the United States.

• Must indicate if employment visa sponsorship is required.

🏝️ Benefits

• No benefits, perks, or additional compensation are specified in the posting.