AI Platform Engineer, Harness

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and sustain enterprise-level AI platform capabilities that support Large Language Models (LLMs), AI agents, Retrieval-Augmented Generation (RAG), and Generative AI applications.

• Create reusable AI frameworks to streamline testing processes, prompt evaluation, model benchmarking, regression testing, and quality assurance.

• Construct AI evaluation frameworks to assess model accuracy, retrieval quality, hallucination detection, latency, throughput, cost, and overall performance of applications.

• Implement observability and monitoring solutions for AI applications, which include telemetry, tracing, logging, dashboards, and operational metrics.

• Develop and maintain LLMOps pipelines that facilitate model deployment, versioning, evaluation, experimentation, rollback, and continuous improvement.

• Design automated workflows for prompt testing, retrieval evaluation, AI system validation, and performance benchmarking.

• Create internal tools for prompt management, model experimentation, AI performance optimization, and enhancing developer productivity.

• Develop scalable backend services and APIs to support AI platforms and enterprise AI integrations.

• Collaborate with AI architects and engineering teams to integrate LLMs, RAG pipelines, vector databases, and agentic AI solutions into enterprise applications.

• Assist in the deployment of AI services across AWS, Azure, or Google Cloud using containerized and cloud-native architectures.

• Implement CI/CD pipelines and infrastructure automation to support enterprise AI development and deployment.

• Enforce security, governance, and Responsible AI controls throughout the AI development lifecycle.

• Assess emerging AI frameworks, LLMOps technologies, evaluation methodologies, and automation tools to enhance engineering productivity.

• Diagnose production AI issues and consistently enhance platform reliability, scalability, security, and user experience.

• Document engineering standards, AI platform architecture, evaluation methodologies, and operational best practices.

⛳️ Requirements

• Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Data Science, or a related technical discipline.

• Over 5 years of experience in software engineering, platform engineering, backend engineering, DevOps, cloud engineering, or infrastructure engineering.

• At least 2 years of experience in building or supporting Generative AI, Large Language Model (LLM), or machine learning applications.

• Proficient programming experience in Python.

• Experience in developing APIs, backend services, and distributed systems.

• Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.

• Experience in deploying applications utilizing Docker and Kubernetes.

• Proficient in working with Git, CI/CD pipelines, Infrastructure as Code (IaC), and infrastructure automation.

• Strong understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Prompt engineering, Embeddings, vector databases, AI agents, and agentic workflows.

• Familiarity with AI evaluation methods, automated testing, benchmarking, regression testing, and model validation.

• Proven experience in building scalable, production-grade software platforms.

• Excellent problem-solving, debugging, and performance optimization abilities.

🏝️ Benefits

• The opportunity to contribute to high-visibility federal missions.

• A culture that promotes innovation, growth, and collaboration.

• Access to state-of-the-art tools and technologies.

• Comprehensive benefits package for you and your family.

• A career path that recognizes ambition and performance.