Remotery

AI Infrastructure Engineer – Agents, ML Systems

atHavocAIRemoteUS flagUnited StatesFull-timeInfrastructure EngineerMid-levelSenior$175k – $200k/year

Posted Aug 5

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop internal AI infrastructure that integrates LLMs and AI agents with various internal tools, APIs, data sources, data lakes, telemetry stores, simulation tools, code repositories, documentation systems, logs, and engineering workflows.

• Design and sustain agentic AI systems aimed at automating tasks, analyzing data, supporting engineering efforts, managing simulation workflows, and enhancing internal productivity.

• Establish tool integration and connector infrastructure for AI agents, including MCP and other upcoming tool-use standards.

• Create pipelines for retrieval, RAG, context management, document processing, embeddings, and internal knowledge search.

• Assist in ML infrastructure workflows encompassing data preparation, dataset curation, experiment tracking, model evaluation, fine-tuning support, and model deployment.

• Develop evaluation frameworks focused on agent performance, tool-use reliability, task success, model quality, regression testing, and failure analysis.

• Create observability, logging, tracing, auditability, monitoring, and debugging tools for AI agents, model calls, MCP tools, and ML pipelines.

• Collaborate with Autonomy, Software, Data, Simulation, Product, and Operations teams to pinpoint AI use cases and construct dependable internal tools.

• Ensure the security of agentic AI systems through least-privilege access, sandboxed execution, prompt-injection mitigation, secrets management, human approvals, and careful handling of sensitive and defense data.

• Maintain documentation, reusable examples, templates, and best practices to support safe AI-tool adoption.


⛳️ Requirements

• Bachelor's degree in Computer Science, Engineering, Machine Learning, Data Science, Applied Mathematics, or a closely related technical field.

• A minimum of 3 years of experience in software engineering, infrastructure engineering, ML infrastructure, backend systems, data engineering, developer tools, or similar technical roles.

• Proficient programming skills in Python, TypeScript, Go, C++, or related languages.

• Proven experience in building production software systems, APIs, services, data pipelines, or internal platforms.

• Familiarity with, or a strong enthusiasm for, LLM applications, AI agents, tool-using systems, RAG pipelines, or AI developer tools.

• Knowledge of embeddings, vector search, prompt management, evaluation, model serving, fine-tuning, or MLOps.

• Competence in integrating APIs, databases, object stores, documents, logs, internal tools, and both structured and unstructured data sources.

• Solid understanding of reliability, observability, testing, and maintainability aspects.

• Strong foundation in securing AI and agentic systems, including least-privilege tool access, prompt-injection and misuse mitigation, secrets management, and safe handling of sensitive data.

• Ability to collaborate effectively across Software, Data, ML, Infrastructure, and Product teams.

• Excellent debugging skills and comfort in working with complex distributed systems.

• U.S. citizenship with the capability to obtain and maintain a security clearance.

• Preferred: experience with MCP, AI tooling, vector databases, embeddings, retrieval, RAG, fine-tuning, Kubernetes, Docker, Ray, Airflow, Dagster, MLflow, Weights & Biases, Kafka, Postgres, S3-compatible storage, internal platforms, workflow automation, human-in-the-loop systems, engineering integrations, autonomy, robotics, simulation, telemetry, perception, defense technology, or secure AI systems.

• An active or prior security clearance is preferred.


🏝️ Benefits

• 100% Employer paid Health, Dental and Vision Insurance for you and your families.

• Life Insurance (Employer Paid).

• Opportunity to participate in the company's 401k program (Matching).

• Unlimited PTO policy with an enforced 2-week minimum.

• Equity Package.

• Work / Home Office Stipend.

• Global Entry.

• 16 Week Paid Parental Leave.

• Monthly Health and Wellness Stipend.

• Bonus offered (compensation summary).

• Remote work arrangement.

People also viewed

Tailscale6 hours ago

Security Infrastructure Engineer

CA flagCanada, +1 more countryFull-timeInfrastructure EngineerC$218.4k – C$273.4k/year
ApplyView job
Adzuna8 hours ago

Infrastructure Engineer

GR flagGreece, +4 more countriesFull-timeInfrastructure Engineer€60k – €70k/year
ApplyView job
Webflow10 hours ago

Staff Software Engineer, Infrastructure

US flagUnited States, +1 more countryFull-timeInfrastructure Engineer$186.7k – $278k/year
ApplyView job
Coinbase11 hours ago

Staff Infrastructure Engineer – Trading

US flagUnited States OnlyFull-timeInfrastructure Engineer$218k – $256.5k/year
ApplyView job
Bishop Fox12 hours ago

Infrastructure Engineer

US flagUnited States OnlyFull-timeInfrastructure Engineer
ApplyView job
Tailscale12 hours ago

Security Infrastructure Engineer

US flagUnited States, +1 more countryFull-timeInfrastructure Engineer$163k – $204k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers