Staff Software Engineer, Foundations

This is a fully remote position, open to applicants in United States.

📋 Description

• Lead the transition of Vanta’s resource data model from a Mongo-centric architecture to a schema-aware solution supported by Postgres, ensuring both iterations operate simultaneously without disrupting customer integrations.

• Facilitate solutions across interdependent engineering teams.

• Design for accuracy under eventual consistency, incorporating idempotent session handling, conditional writes, reconciliation, and explicit backpressure.

• Develop and refine data ingestion and pipeline architecture for dependable, high-throughput processing of terabyte-scale data streams across distributed systems.

• Transform the Query API from an internal utility into a production-ready platform featuring schema versioning, joins, exports, tenant isolation, and predictable latency under load.

• Construct data infrastructure that is externally auditable for SOC 2 or FedRAMP compliance.

• Troubleshoot production issues related to hot partitions, unbounded fan-out, and online modifications of continuously written tables.

• Establish architectural guidance for Kafka, event queuing, Redis caching, Postgres, and MongoDB within the Foundations stack.

• Mentor senior engineers through design evaluations, architectural advice, and hands-on contributions.

• Advocate for reliability, observability, and operational standards across data infrastructure.

• Utilize AI tools and systems to enhance productivity and improve the developed systems.

⛳️ Requirements

• A proven history of leading platform migration projects (v1→v2 architecture transitions) while ensuring backward compatibility and system uptime.

• Strong expertise in streaming and queuing frameworks (Kafka or similar), including partitioning strategies, consumer group dynamics, redelivery, and idempotency.

• Extensive industry experience in distributed systems and data pipeline engineering.

• Demonstrated capability in designing and managing systems at an enterprise scale, including terabyte-scale ingestion and high-throughput event processing.

• Proficient in relational and document data modeling at scale, using Postgres and MongoDB.

• Experience in query planning, connection pooling, and online schema modifications on large, continuously written tables.

• Proven collaboration with engineering leadership to outline multi-quarter technical roadmaps and successfully drive them to completion.

• Strong foundational knowledge of AWS.

• Comfort with TypeScript and Node.js.

• Ability to communicate complex systems trade-offs to both technical and non-technical audiences, facilitating cross-functional collaboration on long-term technical decisions.

• Willingness to leverage AI to enhance skills and improve work outputs, coupled with curiosity, a desire to learn, and sound judgment in the responsible application of AI.

🏝️ Benefits

• Competitive salary and equity package.

• Comprehensive medical, dental, and vision insurance, with 100% coverage of employee-only premiums for most medical plans.

• 16 weeks of paid Parental Leave for all new parents.

• Health and wellness stipend.

• Stipends for remote workspace, internet, and cell phone expenses.

• Commuter benefits for team members reporting to the SF and NYC offices.

• Family planning benefits.

• Matching 401(k) contributions with immediate vesting.

• Flexible PTO policy, along with 80 hours of Sick Time.

• 11 company-paid holidays.

• Virtual team-building activities, lunch-and-learns, and other company-wide events.