QA Engineer, Voice AI Agents

This is a fully remote position, open to applicants in Florida, +1 more state.

📋 Description

• Examine issues reported and identified in live campaigns, such as dropped calls, incorrect agent responses, unsuccessful handoffs, inaccurate scheduling results, and data discrepancies.

• Track problems through call recordings, transcripts, system logs, campaign configurations, and integration data to determine the root causes.

• Create clear and reproducible bug reports that include severity, impact on members, and supporting evidence.

• Prioritize issues and direct them to the appropriate owner.

• Conduct pre-launch quality assurance on campaigns and workflow modifications, testing scripts, edge cases, escalation processes, and handoffs.

• Carry out ongoing conversation quality assurance on active campaigns and identify trends.

• Validate targeting, sequencing, disposition logic, and outcome documentation.

• Assist in developing and maintaining test cases, scoring rubrics, regression suites, and release checklists.

• Automate repetitive tasks such as transcript analysis, configuration verification, and outcome reconciliation.

• Monitor quality metrics and communicate patterns to deployment, product, and engineering teams.

• Collaborate across functions with engineers, deployment managers, and clinical and nursing operations personnel.

⛳️ Requirements

• 3 to 5 years of experience in quality assurance, support engineering, technical operations, or a related role within a production software environment.

• Strong debugging skills.

• Proficient in reading logs, querying data using SQL, and working with APIs along with JSON or YAML configurations.

• Experience in writing test plans and bug reports.

• Excellent written communication and meticulous documentation practices.

• Good judgment regarding severity and prioritization of issues.

• Preferred: experience with voice AI, conversational AI, IVR, or contact center platforms.

• Preferred: scripting skills in Python or a similar programming language.

• Preferred: familiarity with LLM evaluation, prompt testing, or conversation quality scoring.

• Preferred: experience in healthcare, particularly in payer or provider outreach, with knowledge of HIPAA and PHI handling.

• Preferred: experience with observability tools or call analytics tools.

🏝️ Benefits

• Meaningful equity.

• Generous paid time off (PTO).

• Comprehensive health, vision, and dental insurance.

• Life and disability insurance.

• Simple IRA matching.

• Professional development budget for conferences, courses, and certifications.

• Wellness programs.