Senior Software Engineer II – Applied AI

This is a fully remote position, open to applicants in Washington.

📋 Description

• Lead the development of the AI Platform Foundation: Oversee the design and management of the core infrastructure that underpins all Smartsheet AI experiences. Concentrate on creating a resilient, multi-tenant environment that minimizes friction for internal teams, enabling them to deploy dependable and scalable AI features effortlessly.

• Standardize the AI Developer Path: Design high-level abstractions and 'Golden Path' APIs that make AI development accessible across Smartsheet. By shielding product teams from infrastructure complexities, you will empower them to deliver intelligent features swiftly while ensuring safety and consistency at scale.

• Engineer AI Trust & Safety Systems: Create essential monitoring and quality assurance layers that safeguard Smartsheet customers. By developing thorough evaluation pipelines, you will guarantee that every AI-powered feature meets the stringent standards for safety, data privacy, and predictable performance expected by our enterprise partners.

• Drive technical strategy: Collaborate with principal engineers to outline the technical roadmap for Smartsheet’s AI infrastructure, making architectural choices that will influence how we build with AI for years to come.

⛳️ Requirements

• Over 8 years of software engineering experience, including a minimum of 2 years directly working with LLMs in production environments.

• Extensive, hands-on expertise in prompt engineering and context engineering, with a strong understanding of how model behavior varies based on framing, structure, and input design.

• Solid working knowledge of RAG architectures: including chunking strategies, embedding models, retrieval evaluation, and failure diagnosis.

• Experience in building or extending LLM evaluation frameworks; you have designed scoring systems, worked with golden datasets, and have a clear vision of quality standards.

• Proficient in Python, comfortable operating within data-intensive environments (such as Databricks, Delta tables, or equivalents).

• Capable of conveying complex quality findings (both written and verbal) to technical and non-technical stakeholders, explaining issues, their significance, and necessary next steps clearly.

• Strong cross-functional judgment; you know when to escalate issues, when to resolve them independently, and how to establish credibility across engineering, product, and AI platform teams.

• A tendency toward clarity in ambiguous situations; when failure modes are unclear and trade-offs are significant, you provide structure and a clear perspective rather than awaiting consensus.

• Strong Plus: Previous experience in an Applied AI or LLMOps platform within a product company.

• Familiarity with Kubernetes (EKS/GKE): The industry standard for AI, including skills in managing GPU scheduling, auto-scaling based on token throughput, and utilizing tools like Karpenter for cost-efficient node provisioning.

• Experience with Infrastructure as Code (IaC): Using Terraform, Pulumi, or AWS CDK to provision Vector Databases, SQS queues, and S3 buckets.

• Proficient in managing and optimizing Vector Databases such as Pinecone, Milvus, Weaviate, or Databricks Vector Search.

• Experience in building or configuring AI Gateways (like LiteLLM or Kong AI Gateway) to manage rate-limiting, PII masking, and cost-tracking.

• Knowledge of LLM Observability: Setting up tracing tools like Langfuse, LangSmith, or MLflow to monitor 'Time to First Token' (TTFT) and trace hallucination issues.

• Experience with Model-Based Evaluations: Implementing automated scoring systems (like RAGAS or DeepEval) that utilize an 'LLM-as-a-Judge' to assess production outputs.

🏝️ Benefits

• Employer-subsidized medical, vision, and dental coverage for full-time employees.

• 401k Match to support your future savings (50% of your contribution up to the first 6% of your eligible pay).

• Monthly stipend to enhance your work and productivity.

• Flexible Time Away Program, in addition to Sick Time Off.

• US employees automatically receive Smartsheet-sponsored life insurance, short-term, and long-term disability plans.

• US employees are entitled to 12 paid holidays each year.

• Up to 24 weeks of Parental Leave.

• One personal paid Volunteer Day to contribute to our community.

• Opportunities for professional growth and development, including access to Udemy online courses.

• Company-funded perks, such as a counseling membership, local retail discounts, and your own personal Smartsheet account.

• Teleworking options from any registered location in the U.S. (role-specific).