AI Solutions Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Develop, test, deploy, and manage AI-driven workflows, agents, and automation solutions within Salesforce, Microsoft 365, Azure AI, and Copilot.

• Implement prompt logic, orchestration flows, tool invocation, and context retrieval for systems based on LLM.

• Convert approved AI solution designs into scalable, maintainable, and secure implementations.

• Ensure that AI solutions are production-ready, observable, robust, and aligned with business objectives.

• Design and maintain evaluation frameworks for LLM to assess accuracy, relevance, consistency, and impact.

• Execute offline and online evaluations utilizing test datasets, golden answers, regression suites, feedback loops, telemetry, and behavioral signals.

• Establish evaluation thresholds, quality gates, and criteria for launch readiness.

• Formulate strategies to minimize hallucinations through RAG, context filtering, grounding, citations, guardrails, validation, and response constraints.

• Monitor AI outputs, confidence signals, and failure modes within live environments.

• Investigate inaccuracies or low-confidence outputs and implement corrective measures.

• Define and instrument metrics that encompass quality, adoption, latency, reliability, and operational or revenue impacts.

• Create telemetry that links AI usage to cycle-time reduction, capacity enhancement, and risk mitigation.

• Develop context pipelines utilizing structured data, documents, and governed knowledge assets.

• Ensure data quality, access controls, grounding standards, and versioning compliance.

• Assist with documentation, testing, auditability, change management, privacy, security, and responsible AI requirements.

• Collaborate with Product Managers, AI Solutions Architects, Systems teams, analytics teams, and GTM stakeholders.

• Facilitate enablement, adoption, reviews, retrospectives, and ongoing post-launch enhancements.

⛳️ Requirements

• Strong background in software, data, or automation engineering with experience in operating production systems.

• Hands-on experience in building and managing LLM-based workflows and agents.

• Proven experience in designing and operating LLM evaluation frameworks.

• Experience with offline evaluations using test datasets and regression frameworks.

• Experience with online evaluations utilizing user feedback, telemetry, and behavioral signals.

• Experience in minimizing hallucinations in production AI systems through grounding, validation, and guardrails.

• Experience in defining metrics and success criteria.

• Strong understanding of retrieval-augmented generation (RAG), prompt design, and context engineering.

• Familiarity with CRM, GTM tools, and workflow platforms.

• Strong analytical and debugging skills for complex AI system behaviors.

• Excellent written and verbal communication skills for both technical and non-technical audiences.

• Experience in AI engineering, applied machine learning, automation engineering, or similar roles.

• Experience in deploying AI or automation systems into production environments.

• Experience collaborating with cross-functional product, operations, and systems teams.

• Experience in supporting AI quality, reliability, and evaluation in live systems.

• Experience with LLMs, AI agents, prompt engineering, orchestration frameworks, retrieval-augmented generation (RAG), or workflow automation.

• Quality-first mindset that integrates testing, reliability, observability, security, and governance.

• No degree required; Granicus states that most roles do not have degree requirements.

🏝️ Benefits

• Remote-first work environment.

• Employee Resource Groups.

• Coffee with Mark sessions featuring the CEO.

• Microsoft Teams communities focused on wellness, art, pets, family, and parenting.

• Special guest sessions addressing topics that affect employees.