Principal Software Engineer, L5

This is a fully remote position, open to applicants in Ireland.

📋 Description

• Develop and implement significant elements of Twilio's carrier test and observability platform for evaluating active voice and messaging performance, quality assessment, and comprehensive global testing coverage.

• Create high-throughput data and alerting systems that convert data into actionable insights for operational teams.

• Enhance and strengthen production LLM systems for automated troubleshooting of carrier issues.

• Broaden the range of carrier problems addressed without human intervention.

• Oversee evaluation, observability, and safety measures for secure autonomous actions.

• Analyze cross-boundary failures that involve test outcomes, carrier actions, and platform telemetry.

• Implement observability to differentiate between various failure scenarios.

• Participate in technical design and review processes across the team.

• Guide engineers and assist in establishing written standards.

⛳️ Requirements

• 12+ years of experience in software engineering, including substantial time at a staff or principal level working on large-scale systems.

• Extensive expertise in designing, managing, and troubleshooting distributed systems that handle and process high-volume time-series or event data in production, encompassing alerting, anomaly detection, and failure isolation.

• Practical experience in deploying LLM-based systems into production and maintaining them, including context and tool design, performance evaluation against actual outcomes, cost and latency management, and safety measures for actions initiated by model outputs.

• Strong command of at least one backend programming language used at scale: Java, Go, Scala, or a similar language.

• Proficiency with event streaming and cloud infrastructure, including Kafka, AWS, and Kubernetes.

• Ability to clearly articulate complex technical trade-offs in writing and influence colleagues through design reviews.

• Experience in telecommunications, messaging, or voice systems.

• Familiarity with active network testing, monitoring, or quality-of-service measurement.

• Background in agentic systems that perform actions on live infrastructure and evaluation practices for reliable operations.

🏝️ Benefits

• Competitive compensation.

• Generous vacation policy.

• Comprehensive parental and wellness leave.

• Health insurance.

• Retirement savings plan.

• Support for volunteering and charitable contributions.

• Remote-first work policy.

• Occasional in-person team gatherings, functional off-sites, customer meetings, and project or team meetings.