AI Quality and Content Specialist

This is a fully remote position, open to applicants in Canada.

📋 Description

• Conduct audits of AI-generated responses across chat, case management, and self-service interactions.

• Detect hallucinations, assumed resolutions, incomplete answers, and compliance-related risks.

• Evaluate AI performance according to established Quality Assurance frameworks.

• Review and validate knowledge content, workflows, and source materials utilized by AI systems.

• Identify outdated, conflicting, incomplete, or low-confidence information.

• Develop content governance processes for help center articles, macros, internal documentation, and workflow logic.

• Analyze the root causes of AI failures, such as content deficiencies, ineffective prompt management, and edge scenarios.

• Deliver actionable insights to the Content Strategist and Head of Support.

• Create a feedback loop with Product and Engineering teams, representing the voice of the customer.

• Design and maintain dashboards to track resolution accuracy, escalation causes, and customer sentiment.

• Identify trends and propose enhancements.

• Collaborate with stakeholders to retrain AI, enhance knowledge content, and refine workflows and guardrails.

• Assist in establishing a closed-loop AI optimization framework.

• Develop scalable auditing processes to detect content gaps, outdated documentation, and declines in AI confidence or resolution quality.

• Contribute to defining standards for AI-compatible knowledge design.

⛳️ Requirements

• Must reside in Canada.

• Must be eligible for employment without sponsorship.

• Capability to work remotely across various time zones.

• Potential travel to Corporate Headquarters in Denver, Colorado, or other office locations throughout North America may be required.

• Relevant experience and expertise pertaining to the role.

• Proficiency in auditing AI-generated responses across chat, case management, and self-service channels.

• Skill in identifying hallucinations, assumed resolutions, incomplete answers, and compliance risks.

• Ability to audit and validate AI knowledge content, workflows, and source materials.

• Competence in analyzing content gaps, ineffective prompt handling, and edge cases.

• Experience in building and maintaining performance dashboards.

• Ability to collaborate effectively with Content, Product, Engineering, and Support teams.

🏝️ Benefits

• Flexibility to work where and how you prefer within your country of employment – in-office, remote, or hybrid.

• Comprehensive wellness benefits, including an annual wellness stipend.

• Starting with 3 weeks of vacation.

• A paid volunteer day of your choosing.

• 2 floating holidays to celebrate any significant events that matter to you.

• Employee Stock Purchase Program.

• Registered Retirement Savings Plan (RRSP) with a 4% company match.