Senior Test Automation Engineer, Python

This is a fully remote position, open to applicants in Ukraine.

📋 Description

• Develop comprehensive test strategies and create test cases for each use case, encompassing functional, regression, integration, and exception scenarios.

• Conduct structured testing of automation workflows, low-code applications, and AI agent outputs in accordance with acceptance criteria.

• Design and execute agent-specific validation, focusing on accuracy, consistency, and safe failure behavior under edge case conditions.

• Record, prioritize, and monitor defects in collaboration with AI Engineers and the Solution Architect until resolution is achieved.

• Take ownership of the formal QA sign-off prior to release, serving as the ultimate quality checkpoint before Release and Transition.

• Assist with post go-live monitoring in collaboration with the Command Centre managed service, where applicable.

• Contribute quality and defect metrics to the COE methodology's E2E Summary Dashboard for cross-client reporting.

⛳️ Requirements

• Demonstrated experience in QA or test engineering within an automation, low-code, or AI delivery framework.

• Practical experience with Python and Playwright.

• Hands-on knowledge of testing AI or agentic systems, including understanding of non-deterministic output and statistically sound validation methods.

• Familiarity with test management and defect tracking, preferably integrated with Jira.

• Awareness of responsible AI and governance standards for pre-release testing, encompassing bias, safety, and escalation behavior.

• Ability to work efficiently across multiple concurrent use cases within a factory-style delivery rhythm.

• Proficient in clear, organized written communication for defect reporting and sign-off documentation.

• Upper-Intermediate level of English proficiency.

• Nice-to-have: Knowledge of KQL for output and log analysis in Azure Log Analytics.

• Nice-to-have: Familiarity with Microsoft Power Platform and Power Automate.

• Nice-to-have: Understanding of Microsoft Copilot Studio and AI agents within the Microsoft ecosystem.

🏝️ Benefits

• Equal opportunities in recruitment, career advancement, and leadership roles.

• A diverse and inclusive workplace environment.

• Support from a dedicated talent team and access to information regarding employee assistance.