Staff Software Engineer – Experimentation

This is a fully remote position, open to applicants in California, +2 more states.

📋 Description

• Take charge of the platform that Headway utilizes for conducting experiments throughout the organization.

• Create and develop experimentation tools for user assignment, exposure tracking, and cross-device identity resolution.

• Assess and potentially streamline the experimentation and feature-flagging architecture of LaunchDarkly and Statsig.

• Incorporate AI for validating experiment setups and automatically identifying broken experiments.

• Broaden experimentation capabilities to include market-based testing, geographic market assessments, and long-term holdout groups.

• Set defaults, establish review processes, create documentation, and provide training to enhance company-wide experimentation practices.

• Collect requirements from various teams, prioritize them into a roadmap, and oversee the technical product direction of the platform.

• Manage assignment, exposure, and experiment data pipelines.

• Collaborate with Data Science on statistical methodologies and with teams focused on event tracking and metric definitions.

• Lead the technical direction through design documents, reviews, and planning sessions.

• Provide mentorship to senior engineers.

• Work across backend services, front-end rendering, CDN functionality, and the data warehouse.

⛳️ Requirements

• Proven experience in owning an end-to-end experimentation or feature-flagging stack.

• Background in building assignment persistence, exposure tracking, cross-session and cross-device identity resolution, or experiment diagnostics.

• Strong fluency in statistics to collaborate effectively with data scientists and evaluate result validity.

• Experience troubleshooting measurement issues such as uneven traffic splits, users in both experiment arms, exposure discrepancies, and broken identity resolution.

• Demonstrated experience in enhancing organizational experimentation through defaults, templates, reviews, and educational initiatives.

• Proven ability to make build-versus-buy platform decisions and assess vendor risks along with warehouse-native approaches.

• Experience leading migrations by facilitating an easier transition to the new system compared to the old one.

• Capable of leveraging AI while maintaining clear boundaries for human oversight.

• Proven experience directing technical strategies through design documents, reviews, and planning cycles.

• Experience mentoring senior engineers.

• Willingness to engage with backend services, front-end rendering, CDN behavior, and data warehouse systems.

• Familiarity with multi-sided marketplaces, growth or search and ranking systems, or healthcare is advantageous but not mandatory.

• Proficient in Python 3 and TypeScript.

• Experience with FastAPI, SQLAlchemy, React, Remix, LaunchDarkly, Statsig, dbt, Snowflake, Postgres, Redis, AWS, Cloudflare, Spark, Kafka, Datadog, PagerDuty, Sentry, and GitHub.

🏝️ Benefits

• Equity compensation.

• Coverage for Medical, Dental, and Vision.

• HSA / FSA options.

• 401K plan.

• Work-from-Home Stipend.

• Therapy reimbursement.

• 16-week parental leave for eligible employees.

• Annual reimbursement and membership for Carrot Fertility.

• 13 paid holidays each year.

• A holiday break during the week between December 25th and December 31st.

• Flexible PTO policy.

• Employee Assistance Program (EAP).

• Opportunities for training and professional development.

• Comprehensive health and wellness benefits.

• Meaningful ownership opportunities through equity.