Director – AI Product Engineering

Posted 3 days ago

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, test, and deploy production-level AI applications, services, APIs, and reusable engineering components utilizing Python, Node.js, React, and AWS.

• Architect comprehensive LLM applications, which include RAG, prompt and context design, structured outputs, tool utilization, agentic workflows, APIs, and MCP.

• Create application data models and databases with a focus on integrity, access control, and the handling of sensitive data.

• Construct evaluation harnesses, golden datasets, regression suites, and safeguards to prevent hallucination and misuse.

• Oversee production observability concerning latency, token costs, error rates, and output quality.

• Lead assessments on whether to build or buy commercial AI platforms versus internal solutions.

• Implement machine learning techniques for classification, forecasting, and anomaly detection.

• Guide and nurture the AI Product Engineering team.

• Establish standards for transitioning prototypes to production and manage the production handoff to Infrastructure.

• Set engineering benchmarks for architecture, reusable components, code reviews, testing, documentation, and releases.

• Develop AI-specific security measures, including defenses against prompt injection and ensuring least-privilege access.

• Manage CI/CD pipelines for automated builds, testing, security checks, and deployments.

• Supervise offshore engineers and testers to maintain consistent engineering and validation standards.

• Collaborate with IT Security and AI Validation & Quality to ensure regulated applications comply with 21 CFR Part 11 and computer software assurance requirements.

• Track the adoption and business outcomes of deployed tools and report advancements to senior leadership through demonstrations.

• Uphold company values, policies, procedures, and business ethics.

• Perform additional duties as assigned.


⛳️ Requirements

• Bachelor’s degree in Computer Science, Software Engineering, or a related field with equivalent practical experience.

• Over 12 years of professional experience in building and deploying production software.

• At least 3 years of experience in a managerial or technical lead role overseeing engineers.

• Proven experience in managing performance and developing engineers into senior positions.

• Experience managing distributed teams across various time zones, including offshore or vendor engineering and testing teams.

• Strong engineering background in React and Python or Node.js, with recent production coding experience.

• Solid understanding of relational databases and SQL, including schema design, query optimization, and migrations.

• Proficiency with AI-assisted development tools such as Claude, Copilot, or Cursor.

• At least 2 years of experience building and operating production LLM applications on platforms like Anthropic, OpenAI, or similar APIs.

• Extensive hands-on experience with retrieval-augmented generation, including document parsing, chunking, embeddings, vector and hybrid search, and reranking.

• Experience in designing LLM evaluations, golden datasets, automated and LLM-as-judge scoring, and regression testing.

• Background in instrumenting AI applications for production observability, including tracing, cost, latency, and quality monitoring.

• Experience in developing agentic systems that incorporate multi-step tool utilization, planning, and MCP.

• Practical AWS experience in deploying and operating production services, including compute, serverless, storage, and IAM.

• Familiarity with enterprise data platforms such as Redshift or Databricks.

• Understanding of machine learning fundamentals, including classification, forecasting, and anomaly detection.

• Proven history of making AI build-vs-buy decisions based on evaluation with real data.

• Experience in establishing engineering standards, code reviews, testing, CI/CD, release management, and documentation practices.

• Comfort with working with APIs and integrating enterprise systems.

• Exceptional communication skills for executive presentations, scope negotiations, and daily interactions with non-technical stakeholders.


🏝️ Benefits

• Remote work arrangement.

• Minimal travel requirements.

• Commitment to equal-opportunity employment.

• Application accommodations available.

People also viewed

Helpware9 hours ago

Senior Principal Java Software Engineer

MX flagMexico OnlyFull-timeFull-stack Engineer
ApplyView job
Introhive9 hours ago

Software Engineer Contractor

CA flagCanada OnlyFreelanceFull-stack Engineer
ApplyView job
Boeing9 hours ago

Senior Embedded Software Engineer

US flagUnited States OnlyFull-timeFull-stack Engineer$171.7k – $232.2k/year
ApplyView job
M9 Solutions9 hours ago

HCM Technical Lead

US flagUnited States OnlyFull-timeFull-stack Engineer$225k – $260k/year
ApplyView job
Game Plan Tech9 hours ago

Product Engineer

US flagUnited States OnlyFull-timeFull-stack Engineer
ApplyView job
Airbus9 hours ago

Systems & Software Architect

DE flagGermany OnlyFull-timeFull-stack Engineer
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers