QA Engineer

This is a fully remote position, open to applicants in Virginia.

📋 Description

• Take ownership of and uphold the system test plan for functional, integration, performance, and load testing.

• Convert product acceptance criteria into actionable tests.

• Evaluate acceptance criteria prior to development, identifying any ambiguities or unverifiable requirements.

• Ensure accountability among engineering, platform, and partner teams regarding acceptance criteria and release evidence.

• Collaborate independently with engineering to validate delivered work through testing.

• Organize and coordinate customer user acceptance testing, tracking findings to resolution.

• Develop and sustain automated test suites that are integrated into CI/CD pipelines as release gates.

• Set standards for automated coverage and report on compliance with these standards.

• Create performance and load tests that mirror expected user loads.

• Preserve regression coverage throughout releases.

• Construct labeled evaluation sets in collaboration with subject-matter experts.

• Design and implement sampled audits of automated decisions.

• Analyze system misses, correct results, and trends in user overrides.

• Document measurement methods, data, and release evidence comprehensively.

• Generate evidence independently of model builders.

• Maintain an inventory of versions across builds, models, configurations, and thresholds.

• Produce test reports, evaluation records, and rollback criteria for each release.

• Verify that test environments adequately replicate production settings.

• Enforce the release gate to ensure that no release proceeds without the necessary evidence.

⛳️ Requirements

• Over 5 years of experience in software quality engineering, test automation, or software engineering with significant dedicated test ownership.

• A distinct perspective on effective testing practices, along with the judgment to recognize which practices are essential.

• Proficient coding skills in a language and framework suitable for test automation, such as Python with pytest, or JavaScript or TypeScript with Playwright, Cypress, or Selenium.

• Capability to read and understand application code under examination.

• Regular use of AI-assisted development, backed by informed judgment regarding verification processes.

• Proven experience in creating automated test suites and integrating them into CI/CD as release gates.

• Experience managing a test plan through the review and acceptance phases.

• Familiarity with testing against requirements rather than implementations, including collaboration with product owners or end users.

• Background in performance and load testing.

• Working knowledge of statistical literacy, encompassing sampling design and measurement confidence.

• Experience in evaluating system misses, not just successful outcomes.

• Readiness to report unfavorable results independently.

• Ability to gate work conducted by individuals who do not report directly to you.

• Clear and audit-ready written documentation.

• Willingness to uphold and articulate release decisions.

• US Citizenship Required.

• Active US Secret clearance.

• Favorable investigation and CAC eligibility from the outset.

• Willingness to travel up to 25% to customer sites, DEFCON AI HQ, and vendor facilities.

• Elevated security requirements apply to certain aspects of this work.

• Preferred: active Top Secret clearance; experience in government DevSecOps, RMF, or ATO; formal accreditation or audit evidence; experience with contract and API testing; test data management; test infrastructure; AI-assisted testing; familiarity with JMeter, k6, Locust, or similar; machine learning/model-output evaluation; model cards; NIST AI RMF; Section 508; WCAG verification.

🏝️ Benefits

• Competitive salary, bonus, and equity package.

• 100% employer-paid, comprehensive health insurance that includes medical, dental, and vision coverage for you and your family.

• Unlimited PTO, subject to your manager’s approval.

• Flexible work environment allowing you to manage your daily schedule.

• 14 weeks of fully-paid parental leave.

• A fully remote, results-oriented work environment.