Senior QA Automation Engineer, Python, Selenium, Pytest

This is a fully remote position, open to applicants in Costa Rica.

📋 Description

• Develop a comprehensive testing architecture by planning unit, integration, end-to-end, and semantic validation test suites that encompass the entire pipeline from profile intake to roadmap generation; establish coverage targets and acceptance criteria.

• Create semantic validation harnesses for AI outputs by designing golden-set test cases (5–8 CISO-approved reference profiles) that assess AI stage precision/recall, rigor-tier correctness, floor-control invariants, traceability completeness, and run-to-run stability; manage semantic regression blocks merge.

• Execute access-control testing by designing automated probes at the API and database layers to confirm cross-tenant isolation; simulate compromised sessions and ensure that RLS policies enforce tenant boundaries.

• Design and maintain CI/CD quality gates by developing automated test runners that operate on every commit, including unit tests, integration tests, schema validation, migration guards, image scanning, and access-control probes; ensure red builds block merging.

• Lead test automation initiatives by expanding automation frameworks and methodologies; identify areas for enhancing testing processes; recommend tools and technologies to improve testing efficiency.

• Collaborate with the team by acting as the QA representative in technical design reviews; identify testability concerns early; work alongside the AI/LLM Lead to clarify what "correct" means for each stage of the pipeline.

• Mentor junior test engineers by providing guidance on test design, semantic validation, and CI/CD best practices; share insights on testing AI systems where reproducibility is crucial.

• Manage test documentation and reporting by keeping accurate test plans, results, and traceability matrices; report on test coverage and quality metrics to leadership; communicate any blockers and risks transparently.

⛳️ Requirements

• 5+ years of hands-on software QA experience with a proven record in:

• Test strategy, planning, execution, and maintenance.

• Comprehensive test documentation and traceability.

• Advanced test automation (custom frameworks, advanced scripting).

• 3+ years of experience with automated testing tools such as Selenium, Pytest, or equivalent; capability to write custom testing frameworks.

• Advanced proficiency in a relevant programming language for test automation:

• Python (primary for this role; Pytest, async testing).

• Or Java, C#, Go.

• Extensive knowledge of databases and SQL to execute complex data validation, verify multi-tenant isolation at the database layer, and test RLS policies.

• CI/CD pipeline expertise to understand how tests integrate into automated deployment workflows; experience in optimizing test execution for speed and reliability.

• Familiarity with Agile/Scrum methodologies to drive QA strategies in fast-paced, iterative development cycles.

• Proficiency in defect tracking tools (Jira or equivalent).

• Demonstrated leadership skills; mentor junior engineers, advocate for quality standards, and drive process enhancements.

• Outstanding communication skills; able to convey complex technical issues to engineers, product teams, and stakeholders.

• Self-motivated and detail-oriented; capable of working independently under time constraints and managing multiple priorities.

• Exceptional English proficiency; able to collaborate effectively with distributed global teams.

🏝️ Benefits

• Medical Insurance benefits as per company policy.

• Life Insurance as per company policy.

• 12 days of paid vacation, sick leave, and paid holidays in accordance with local law.

• Paternity and maternity leave as per local law.

• Leave for marriage, bereavement, and graduation as per company policy.

• Sick leave and paid holidays as per local law.

• Christmas bonus as per local law.

• Saving Fund “Asociacion Solidarista”.

• Beach house (Managed by the Asociacion Solidarista).

• Discounts in local shops.

• Direct deposit of payroll.

• Paid professional certifications.