AI Agentic Tester

This is a fully remote position, open to applicants in Missouri.

📋 Description

• Conduct functional testing for AI agent workflows and comprehensive AI applications.

• Assess LLM responses for their accuracy, relevance, consistency, and rates of hallucination.

• Evaluate RAG pipelines, prompt engineering, and orchestration of AI agents.

• Perform API testing utilizing Postman, Swagger, and REST APIs.

• Create and sustain automation scripts using Python, Selenium, Playwright, or PyTest.

• Verify interactions between multiple agents, tool invocation, and workflow processes.

• Implement AI testing methodologies and evaluation metrics to guarantee response quality.

• Collaborate with Azure OpenAI, AWS Bedrock, Gemini, or equivalent AI platforms to validate AI solutions.

⛳️ Requirements

• Proficiency in AI Agentic Testing

• Experience in Functional Testing

• Knowledge of Generative AI Testing

• Familiarity with Large Language Models (LLMs)

• Understanding of AI Agents & Agentic AI

• Expertise in Retrieval-Augmented Generation (RAG)

• Skills in Prompt Engineering Validation

• Competence in AI Model Validation & Evaluation

• API Testing experience (Postman, REST APIs, Swagger)

• Proficient in Python

• Test Automation skills (Selenium / Playwright / PyTest)

• Understanding of AI Evaluation Metrics (Accuracy, Hallucination, Relevance, Consistency)

• Experience with End-to-End Workflow Testing

• Multi-Agent Testing capability

• Familiarity with Azure OpenAI / AWS Bedrock / Gemini

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Opportunities for professional development and continuing education.

• Flexible working hours and remote work options.

• Collaborative and innovative work environment.