Staff Software Engineer, Backend – Observability

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of the architecture for Affirm’s telemetry pipeline, ensuring a balance between reliability, query performance, and cost at over 200 TB/day.

• Develop log-sampling techniques to minimize volume and expenses while maintaining incident signals.

• Influence observability practices for agentic development, enabling AI coding agents and code reviews to identify gaps in instrumentation.

• Establish the technical strategy for the team with a long-term vision spanning one year.

• Lead cross-functional, business-critical initiatives in collaboration with product management, design, and analytics teams.

• Define and promote technical solutions along with operational processes.

• Oversee team operations and availability, which includes monitoring, triage rotations, playbooks, policies, testing, alerting, and on-call responsibilities.

• Set standards for code reviews and design, advocating for these through written documentation and technical presentations.

• Mentor and develop a team of six engineers through constructive feedback, guidance, and leading by example.

⛳️ Requirements

• A minimum of 8 years of experience in designing, developing, and launching scalable backend systems using programming languages such as Python or Kotlin.

• Significant experience in creating highly available distributed systems.

• Proficient with AWS, MySQL, Flink, and Kubernetes.

• Experience in building or managing high-volume telemetry systems utilizing metrics, logs, and/or traces.

• Familiarity with OpenTelemetry, Kafka, and Terraform.

• Understanding of structured and wide events, sampling techniques, cardinality management, and Service Level Objectives (SLOs).

• Proven track record of promoting platform standards across engineering teams.

• Experience in delivering major features, system components, or phasing out existing functionalities through comprehensive technical and execution plans.

• Ability to produce high-quality, readable code.

• Capability to navigate low-level language idioms and large system architectures.

• Strong skills in gathering feedback and iterating on solutions.

• Excellent verbal and written communication abilities.

• Equivalent practical experience or a Bachelor's degree in a relevant field.

🏝️ Benefits

• Equity rewards may be part of the total compensation package.

• Monthly stipends available for health, wellness, and technology expenses.

• Comprehensive medical coverage at no cost for employees and their dependents.

• Dental and vision insurance for employees and their dependents.

• Flexible time off policy.

• Generous holiday schedule.

• Employee stock purchase plan (ESPP) that enables employees to purchase Affirm stock at a discounted rate.

• Inclusive interview process with accommodations for candidates with disabilities.

• Remote-first work flexibility.

• In-person onboarding experience provided for all new hires.