
Forward Deployed Engineer, AI Trust and Reliability
Posted 11 hours ago

Posted 11 hours ago
This is a fully remote position, open to applicants in Canada.
• Customize guardrails to align with each partner’s policies, regulations, risk appetite, and languages.
• Develop safeguards for sensitive data and actions, including data masking, identity verification, and obtaining customer consent.
• Adjust guardrails to mitigate real risks while allowing legitimate customers access.
• Conduct red-team assessments for every deployment prior to launch and following significant changes.
• Create and sustain automated adversarial testing suites that operate continuously in every partner language.
• Translate findings into engineering solutions and regression tests.
• Oversee partner security questionnaires, architecture evaluations, AI/model-risk assessments, and remediation of penetration tests.
• Document AI data handling, access protocols, and decision-making processes for regulators and auditors.
• Manage go-live checklists that encompass functional, regression, safety, and load testing, including sign-off criteria and rollback strategies.
• Expand automated testing across products for modifications involving models, prompts, and configurations.
• Facilitate partner user acceptance testing and resolve issues prior to launch.
• Direct the security and privacy facets of partner data access, including agreements, IT assessments, and secure access to partner systems.
• Define and uphold standards for partner data management, including handling, storage, separation, and deletion across deployments and model training.
• Supervise live deployments for quality, reliability, abuse, and safety failures; establish dashboards and alert systems.
• Lead the response efforts to safety or security incidents and maintain communication with partners.
• Collaborate with Research, Product, and Security teams to integrate security measures into new models and features.
• Convert deployments into reusable guardrails, testing protocols, and documentation.
• 5+ years of experience in security engineering, application security, AI safety, quality engineering, or site reliability, with practical involvement in securing and maintaining production systems.
• Experience in red-teaming LLM applications or agents, encompassing jailbreaks, prompt injection, data exfiltration, and tool and permission misuse.
• Designed and optimized guardrails or safety mechanisms for an LLM-based product.
• Acted as a vendor representative in enterprise security assessments, including questionnaires, architecture evaluations, and penetration test remediation.
• Held responsibility for testing and release quality within a production system, from automated test suites and release checkpoints to load testing and rollback processes.
• Proficient in Python with the capability to develop automated test harnesses and tools.
• Background in securing or operating systems within travel or financial service sectors, such as airlines, hotels, OTAs, banks, or payment companies.
• Ability to clearly communicate risk to engineers, partner security teams, and non-technical executives.
• Willingness to travel for launches and reviews.
• Practical experience in fine-tuning models will be evaluated in the application process.
• Experience embedded with an enterprise customer or partner is mandatory for the application.
• Working hours should overlap with 9 am to 5 pm Eastern time.
• Willingness to travel 30 to 50 percent of the time, including extended on-site periods with partners.
• Bonus: Experience in security or risk management within regulated sectors such as banking, payments, or aviation, or conducting model-risk and AI governance reviews at a financial institution.
• Bonus: Knowledge of OWASP Top 10 for LLM applications, SOC 2, ISO 27001, PCI DSS, and NIST AI RMF.
• Bonus: Experience in data privacy and governance practices, including data-sharing agreements, secure access to customer systems, or compliance with GDPR and similar regulations.
• Bonus: Familiarity with observability and incident management tools, such as Datadog, and participation in on-call duties and post-mortems.
• Bonus: Experience in multilingual safety testing, or security in voice and conversational AI.
• Bonus: Experience in forward-deployed or customer-facing security engineering roles.
• Competitive salary along with potential pre-IPO equity packages.
• 100% of premiums for the group insurance plan are covered by Hopper.
• Coverage for life, short-term, and long-term disability.
• Health Savings Account (HSA) covering eligible medical and dental expenses.
• Access to Dialogue’s telemedicine services anytime, anywhere.
• RRSP plan with automatic pre-tax contributions per pay period.
• Generous parental leave policy, exceeding industry standards.
• Unlimited paid time off (PTO).
• Travel stipend provided through Carrot Cash.
• On-demand access to co-working spaces via FlexDesk.
• Work-from-home stipend available.
• An entrepreneurial culture promoting innovation and risk-taking as part of daily operations.
• Open communication channels with management and leadership.
• Small, agile teams yielding significant impacts.
• Equity options, sign-on bonuses, and other potential incentives.
Extend
OhioHealth
Capital One
Software Mind
Get handpicked remote jobs straight to your inbox weekly.