
Forward Deployed Engineer, AI Trust and Reliability
Posted 11 hours ago

Posted 11 hours ago
This is a fully remote position, open to applicants in United States.
• Customize AI guardrails to align with each partner's policies, regulations, risk tolerance, and languages.
• Develop safeguards for sensitive information and actions, which include data masking, identity verification, and customer consent protocols.
• Adjust guardrails to mitigate actual risks while ensuring legitimate customers are not hindered.
• Conduct red-team testing prior to deployment and following significant modifications.
• Create and sustain automated adversarial testing frameworks across various supported languages.
• Translate findings into engineering solutions and regression testing protocols.
• Oversee partner security questionnaires, architecture assessments, and AI/model risk evaluations.
• Document AI data management, access, and decision-making processes for compliance with regulators and auditors.
• Manage deployment go-live checklists, sign-off requirements, and rollback strategies.
• Broaden automated testing across products as well as modifications to models, prompts, and configurations.
• Facilitate partner user acceptance testing and resolve launch-related issues.
• Direct security and privacy efforts regarding partner data access, encompassing agreements, IT evaluations, and secure system access.
• Establish and enforce data handling, storage, separation, and deletion protocols for partners.
• Supervise live deployments to ensure quality, reliability, and safety while monitoring for abuse or failures.
• Lead responses to safety or security incidents.
• Collaborate with Research, Product, and Security teams to ensure secure model and feature launches.
• Transform insights from deployments into reusable guardrails, tests, and documentation.
• Over 5 years of experience in security engineering, application security, AI safety, quality engineering, or site reliability.
• Practical experience in red-teaming LLM applications or agents, covering jailbreaks, prompt injections, data exfiltration, and tool and permission misuse.
• Proven background in designing and fine-tuning guardrails or safety measures for LLM-based products.
• Demonstrated experience as a vendor representative in enterprise security assessments, such as questionnaires, architecture evaluations, and penetration test follow-ups.
• Experience in managing testing and release quality for production systems, including automated testing suites, release gates, load testing, and rollback mechanisms.
• Proficient in Python programming.
• Background in securing or managing systems for travel or financial service organizations.
• Knowledge of fraud, account takeover, and payment-related risks.
• Ability to communicate risk effectively to engineers, partner security teams, and non-technical executives.
• Willingness to travel for launches and reviews.
• Competitive salary.
• Pre-IPO equity packages.
• Unlimited paid time off (PTO).
• Carrot Cash travel stipend.
• Access to co-working spaces on demand through FlexDesk.
• Work-from-home stipend.
• Generous parental leave policy.
• 100% employer-covered Medical, Dental, and Vision insurance for employees.
• Disability and Life insurance options.
• Health Reimbursement Account (HRA).
• DCA/FSA options.
• Access to a 401k retirement plan.
Extend
OhioHealth
Capital One
Software Mind
Get handpicked remote jobs straight to your inbox weekly.