Staff Software Engineer, Backend – Observability

This is a fully remote position, open to applicants in Canada.

📋 Description

• Take charge of the architecture for Affirm’s telemetry pipeline, ensuring a balance between reliability, query performance, and cost efficiency at over 200 TB/day.

• Develop log-sampling strategies that minimize volume and expenses while maintaining incident signal integrity.

• Influence observability practices tailored for agentic development, enabling AI coding agents and code reviews to pinpoint instrumentation gaps.

• Establish the technical direction for the team with a long-term vision spanning a year.

• Lead cross-functional projects that have a significant business impact, collaborating with product, design, and analytics teams.

• Define and promote technical solutions and operational processes.

• Oversee team operations and availability, which includes monitoring, triage rotations, playbooks, policies, testing, alerting, and on-call support.

• Set and advocate for code-review and design standards through writing and tech presentations.

• Mentor and nurture a team of six engineers by providing constructive feedback and guidance.

• Outline the Observability roadmap and act as its technical lead.

⛳️ Requirements

• Minimum of 8 years of experience in designing, developing, and launching scalable backend systems.

• Proficiency in programming languages such as Python or Kotlin.

• Extensive experience in developing distributed systems that are highly available.

• Familiarity with AWS, MySQL, Flink, and Kubernetes.

• Experience in building or managing high-volume telemetry systems using OpenTelemetry, Kafka, and Terraform.

• Understanding of contemporary observability practices, including structured and wide events, sampling techniques, cardinality management, and SLOs.

• Proven track record of driving the adoption of platform standards across engineering teams through tools and engagement initiatives.

• Experience in delivering major features or system components, or phasing out existing functionalities, through effective technical and execution planning.

• Capability to produce high-quality, comprehensible code.

• Ability to navigate from low-level language idioms to large-system architecture.

• Excellent verbal and written communication skills for effective collaboration with a global engineering team.

🏝️ Benefits

• Monthly stipends for health, wellness, and technology expenditures.

• Complete medical coverage for employees and their dependents at no cost.

• Dental and vision insurance for employees and their dependents.

• Flexible time-off policies.

• Generous holiday schedules.

• Employee stock purchase plan (ESPP) with discounted Affirm stock.

• Remote-first work flexibility.

• In-person onboarding experience.

• Inclusive interview process with accommodations for candidates with disabilities.