
Director – AI Product Engineering
Posted 3 days ago

Posted 3 days ago
This is a fully remote position, open to applicants in United States.
• Design, develop, test, and deploy production-level AI applications, services, APIs, and reusable engineering components utilizing Python, Node.js, React, and AWS.
• Architect comprehensive LLM applications, which include RAG, prompt and context design, structured outputs, tool utilization, agentic workflows, APIs, and MCP.
• Create application data models and databases with a focus on integrity, access control, and the handling of sensitive data.
• Construct evaluation harnesses, golden datasets, regression suites, and safeguards to prevent hallucination and misuse.
• Oversee production observability concerning latency, token costs, error rates, and output quality.
• Lead assessments on whether to build or buy commercial AI platforms versus internal solutions.
• Implement machine learning techniques for classification, forecasting, and anomaly detection.
• Guide and nurture the AI Product Engineering team.
• Establish standards for transitioning prototypes to production and manage the production handoff to Infrastructure.
• Set engineering benchmarks for architecture, reusable components, code reviews, testing, documentation, and releases.
• Develop AI-specific security measures, including defenses against prompt injection and ensuring least-privilege access.
• Manage CI/CD pipelines for automated builds, testing, security checks, and deployments.
• Supervise offshore engineers and testers to maintain consistent engineering and validation standards.
• Collaborate with IT Security and AI Validation & Quality to ensure regulated applications comply with 21 CFR Part 11 and computer software assurance requirements.
• Track the adoption and business outcomes of deployed tools and report advancements to senior leadership through demonstrations.
• Uphold company values, policies, procedures, and business ethics.
• Perform additional duties as assigned.
• Bachelor’s degree in Computer Science, Software Engineering, or a related field with equivalent practical experience.
• Over 12 years of professional experience in building and deploying production software.
• At least 3 years of experience in a managerial or technical lead role overseeing engineers.
• Proven experience in managing performance and developing engineers into senior positions.
• Experience managing distributed teams across various time zones, including offshore or vendor engineering and testing teams.
• Strong engineering background in React and Python or Node.js, with recent production coding experience.
• Solid understanding of relational databases and SQL, including schema design, query optimization, and migrations.
• Proficiency with AI-assisted development tools such as Claude, Copilot, or Cursor.
• At least 2 years of experience building and operating production LLM applications on platforms like Anthropic, OpenAI, or similar APIs.
• Extensive hands-on experience with retrieval-augmented generation, including document parsing, chunking, embeddings, vector and hybrid search, and reranking.
• Experience in designing LLM evaluations, golden datasets, automated and LLM-as-judge scoring, and regression testing.
• Background in instrumenting AI applications for production observability, including tracing, cost, latency, and quality monitoring.
• Experience in developing agentic systems that incorporate multi-step tool utilization, planning, and MCP.
• Practical AWS experience in deploying and operating production services, including compute, serverless, storage, and IAM.
• Familiarity with enterprise data platforms such as Redshift or Databricks.
• Understanding of machine learning fundamentals, including classification, forecasting, and anomaly detection.
• Proven history of making AI build-vs-buy decisions based on evaluation with real data.
• Experience in establishing engineering standards, code reviews, testing, CI/CD, release management, and documentation practices.
• Comfort with working with APIs and integrating enterprise systems.
• Exceptional communication skills for executive presentations, scope negotiations, and daily interactions with non-technical stakeholders.
• Remote work arrangement.
• Minimal travel requirements.
• Commitment to equal-opportunity employment.
• Application accommodations available.
Helpware
Introhive
Boeing
M9 Solutions
Get handpicked remote jobs straight to your inbox weekly.