Software Engineer I – LLM Observability

This is a fully remote position, open to applicants in United States.

📋 Description

• Provide end-to-end AI observability capabilities, from data ingestion to in-product visualizations.

• Contribute to the data model and infrastructure necessary for monitoring agents and their ecosystems within the Honeycomb product.

• Develop systems that analyze and highlight AI-specific signals, including cost, token utilization, errors, and anomalies.

• Collaborate with product and design teams to create user-friendly, intuitive features that enhance meaningful user workflows.

• Share insights about exploring and comprehending AI system behavior across various discussions and workloads.

• Engage in rapid feedback cycles and experimentation, communicating both successes and setbacks.

• Operate within a small, cross-functional engineering team alongside a dedicated product manager and designer.

• Deliver features that customers utilize in production environments.

• Aid in defining how individuals understand and troubleshoot AI systems.

⛳️ Requirements

• A minimum of 3 years of software engineering experience.

• Proven experience in delivering customer-facing features at a SaaS company, rather than solely internal tools.

• Proficient in full-stack development, including backend technologies (Go and/or Python) and frontend technologies (TypeScript and React).

• Demonstrated history of collaboration with product managers and designers.

• Regular, hands-on experience with AI tools in daily engineering tasks.

• Experience with on-call responsibilities; ability to participate in the rotation after approximately 90 days.

• Proven ability to navigate uncertainty in a rapidly evolving field where requirements frequently change and experimentation is essential.

• Must verify identity and work eligibility.

• Honeycomb is currently unable to sponsor or support visa transfers.

• Bonus: Experience in building LLM or agent-driven features for production.

• Bonus: Practical experience with observability tools such as OpenTelemetry.

🏝️ Benefits

• Generous equity through an employee-friendly stock program.

• Transparent compensation based on levels relative to experience.

• Unlimited paid time off (PTO).

• A distributed-first culture and mindset.

• Stipend for home office, co-working, and internet expenses.

• Comprehensive benefits coverage for employees, with additional options available for dependents.

• Up to 16 weeks of paid parental leave, regardless of the path to parenthood.

• Annual development allowance.

• Reasonable accommodations provided during the hiring process.