Principal AI Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design AI systems that drive content generation, business intelligence, recommendations, and agentic workflows.

• Develop reusable prompts, evaluation flows, orchestration layers, and architectural patterns across various products.

• Take the lead in agent design, which includes reasoning, tool/API calls, state management, checkpointing, handoffs, and safe failures.

• Create comprehensive RAG pipelines that encompass chunking, embeddings, metadata, reranking, and retrieval evaluation.

• Oversee model selection based on factors such as cost, latency, accuracy, and reliability, including fallback and model-switching strategies.

• Establish evaluation frameworks and conduct regression testing to assess output quality, edit ratios, hallucination rates, schema adherence, latency, failure rates, and inference costs.

• Collaborate with engineering and platform teams to develop deployable, observable, and maintainable systems.

• Package services utilizing Python, FastAPI/Flask, and Docker when applicable.

• Troubleshoot AI-specific production challenges, including rate limits, cost spikes, model failures, and degraded output.

• Mentor GenAI engineers and review their designs, prompts, workflows, and evaluations.

• Convert product requirements into architecture with measurable acceptance criteria.

• Articulate technical trade-offs to engineers and leadership.

⛳️ Requirements

• 5+ years of experience in engineering, AI, ML, or data product roles.

• 3+ years of practical experience with GenAI/LLM systems.

• Proven experience with production GenAI systems handling real traffic.

• In-depth knowledge of LLMs and SLMs, prompt engineering, structured outputs, and tool/function calling.

• Hands-on experience with at least two major LLM ecosystems.

• Experience in building production RAG systems utilizing vector stores like Chroma, Pinecone, Weaviate, or FAISS, alongside embeddings, semantic search, and assessed retrieval quality.

• Practical knowledge of at least one agentic framework such as LangGraph, CrewAI, AutoGen, or Semantic Kernel.

• Familiarity with agent memory, state management, tool integration, and failure handling.

• Proficient in Python.

• Comfortable working with REST APIs, Docker, a cloud platform, and basic CI/CD practices.

• Demonstrated expertise in evaluation work, including regression testing, hallucination checks, schema validation, and output scoring.

• Capability to define metrics for quality, reliability, cost, and business impact.

• Track record of leading a small team or managing AI architecture from inception to completion.

• Skill in establishing structure in a dynamic environment with evolving requirements.

• Preferred: experience with fine-tuning or supervised training workflows.

• Preferred: deployment of SLM and open-source models using vLLM, Ollama, or TensorRT-LLM.

• Preferred: involvement in multimodal work across text, image, audio, or video.

• Preferred: experience with Kubernetes or serverless deployment.

• Preferred: familiarity with evaluation/observability tools like LangSmith, MLflow, or Weights & Biases.

• Preferred: background in marketing technology, SMB intelligence, or content automation.

• Preferred: knowledge of responsible AI, privacy, security, and compliance practices.

• Preferred: experience in scaling high-volume AI systems.

🏝️ Benefits

• Fully remote work environment.

• A production GenAI foundation already handling real volume.

• True architectural ownership of the upcoming agentic systems.

• A small, high-context team that acts swiftly and engages in meaningful discussions about the work.

• Opportunity to make a significant impact on small businesses.