Software Engineer – AI Systems, Go

This is a fully remote position, open to applicants in Arizona, +16 more states.

📋 Description

• Design, create, test, launch, and manage production-quality software and backend services, primarily utilizing Go (Golang).

• Architect and develop AI agents and agent-based systems leveraging tools, APIs, state, and context for intricate multi-step workflows.

• Construct resilient workflows featuring checkpointing, resumption, retries, recovery, and safe continuation following partial failures or interruptions.

• Define roles across models, agents, tools, APIs, data sources, services, and deterministic application logic.

• Design integrations among various AI model providers, encompassing routing, failover, rate-limit management, health monitoring, and degradation strategies.

• Implement safeguards such as structured-output validation, error handling, retries, quality gates, evidence validation, and automated recovery.

• Formulate testing strategies for non-deterministic AI behavior using fixtures, recorded/replayed interactions, regression suites, evaluation harnesses, and quality baselines.

• Enhance observability through tracing, metrics, logging, evaluation data, and replayable execution histories.

• Create human-in-the-loop workflows for review, approval, intervention, and modification of workflow states.

• Develop and maintain APIs, backend services, relational data models, job-processing frameworks, and application components.

• Establish secure ingestion and processing methods for user-supplied documents and external content.

• Optimize reliability, latency, throughput, scalability, output quality, and execution cost.

• Create reusable patterns and shared components for agents, providers, tools, workflows, and output types.

• Take ownership of technical challenges from investigation and architecture to implementation, deployment, monitoring, troubleshooting, and production operation.

• Collaborate with product stakeholders and domain experts to transform qualitative requirements into measurable behaviors and technical solutions.

• Contribute to architectural decisions and engineering standards for the University’s AI-powered applications.

• Establish foundational architecture and engineering practices for Stanbridge University’s broader AI capabilities.

⛳️ Requirements

• Significant professional experience in developing and operating production software, with strong expertise in Go (Golang) or a demonstrated depth in another backend language with a quick adaptability to Go.

• Practical experience in deploying an LLM-backed application, AI agent, or agent-based system into production and providing post-deployment support.

• Strong understanding of AI application architecture, including the interaction between models, agents, tools, APIs, data sources, services, and application logic.

• Demonstrated knowledge of distributed systems engineering, including concurrency, asynchronous processing, queues, idempotency, retries, partial failures, state management, and recovery.

• Experience in designing systems that maintain reliability even when individual services, providers, or workflow stages encounter failures.

• Proven discipline in testing systems that involve non-deterministic behavior.

• Extensive experience in designing and utilizing HTTP APIs.

• Proficiency in relational databases, SQL, and persistent application state.

• Experience in building, deploying, monitoring, and troubleshooting backend services within production environments.

• Understanding of software architecture, testing, debugging, observability, and production engineering practices.

• Ability to independently manage ambiguous technical challenges from investigation to production implementation.

• Strong analytical judgment with the ability to balance reliability, quality, performance, complexity, and cost considerations.

• Preferred: Experience with AI agent frameworks, orchestration patterns, or custom agent architectures.

• Preferred: Experience with retrieval-augmented generation (RAG), embeddings, vector databases, semantic search, or knowledge-retrieval architectures.

• Preferred: Experience in developing evaluation and observability systems for LLM applications.

• Preferred: Experience with judge models, structured-output validation, evidence checking, or other AI quality-control mechanisms.

• Preferred: Experience in designing human-in-the-loop workflows.

• Preferred: Experience with document processing, headless-browser rendering, text-to-speech, image generation, video generation, or other media pipelines.

• Preferred: Experience with event-driven architectures, durable job queues, and asynchronous processing at scale.

• Preferred: Experience with containers, cloud infrastructure, CI/CD, and production deployment environments.

• Preferred: Understanding of prompt injection, authorization boundaries, data isolation, and security considerations.

• Must reside in an eligible state at the time of employment.

• Applicants must be legally authorized to work in the United States.

• Stanbridge University does not provide employment-based immigration sponsorship.

• Official transcripts are required prior to hire.

• Degrees earned outside the United States must be evaluated by a recognized credential evaluation service.

• A job-related assessment may be required during the interview process.

• Employment verification will be conducted to confirm work experience.

• Offers of employment are contingent upon successful completion of a background check.

• Must be able to satisfactorily perform each essential duty and be physically present in the office unless noted otherwise.

• Ability to meet specified physical demands, including sitting for extended periods and lifting, carrying, and/or moving objects weighing 10–25 pounds as necessary.

🏝️ Benefits

• Health Care Plan (Medical, Dental & Vision)

• Retirement Plan (401k)

• Exciting university events

• Seasonal motivational health and wellness challenges

• Work/Life Balance initiatives

• Onsite wellness program / Staff Chiropractor

• Life Insurance (Basic, Voluntary & AD&D)

• Paid Time Off (Vacation, Sick & Public Holidays)

• Family Leave (Maternity, Paternity)

• Reasonable accommodations may be made to enable individuals with disabilities to perform essential functions.