Senior Software Engineer, Data Platform

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and manage distributed services and computing infrastructure for the data platform, which includes containerized workloads on AWS ECS, autoscaling, resource allocation, and orchestration.

• Take full ownership of the design and development of scalable data pipelines and services for data movement, from ingestion and orchestration to transformation and delivery.

• Enhance the flow of event-sourced data from production systems into the data infrastructure and facilitate its return to other systems.

• Participate in on-call duties for the data platform and manage production issues in distributed data systems from start to finish.

• Protect participant and employer data through PII classification and masking, role-based access controls, least-privilege patterns, encryption, key management, lineage, and audit trails to meet SOC 2 and regulatory requirements.

• Guide the technical direction and advancement of the data platform towards an AI-first infrastructure, focusing on AI consumption, unstructured data access, AI tools, and production model and AI workloads.

• Create interfaces for data engineers, analysts, and AI systems, along with systems that transfer curated data from the warehouse into production services and downstream tools.

• Mentor engineers and analysts to elevate standards for testing, code reviews, and operational readiness.

⛳️ Requirements

• A minimum of 5 years of experience in building and managing production software systems, with substantial expertise in data-intensive systems such as data pipelines, data streaming platforms, and data infrastructure.

• Proven experience in designing distributed systems and services, including aspects like concurrency, backpressure, idempotency, partial failure, and associated trade-offs.

• Hands-on experience in operating containerized workloads in production on AWS, including scaling, resource allocation, and performance and cost optimization under real-load conditions.

• Capability to independently manage and enhance complex production systems.

• Familiarity with automated testing, code reviews, CI/CD, and infrastructure as code tools such as Terraform.

• Experience with large-scale workflow orchestration, including tools like Airflow or their equivalents.

• Strong enthusiasm for utilizing AI tools and workflow automation as the primary means of accomplishing tasks.

• Preferred: direct experience with event-sourced or append-only log systems, change data capture, or streaming platforms like Kafka or Kinesis.

• Preferred: working knowledge of cloud data warehouses like Snowflake, including access control, performance optimization, and cost management.

• Preferred: experience in deploying and managing machine learning models in production settings.

• Preferred: background in data lakehouse architectures and open table formats such as Apache Iceberg or Delta Lake.

• Preferred: experience in building data infrastructure for AI-driven data access, including MCP, AI data governance, and assessment of AI-generated query responses.

• Preferred: experience in the fintech sector, financial services, or another regulated or compliance-focused industry.

🏝️ Benefits

• 401(k) plan with a dollar-for-dollar employer match up to 4% of compensation, immediately vested, with no plan fees.

• Comprehensive health plans, dental insurance, and vision coverage.

• Competitive time off policies and parental leave.

• Addition Wealth: Unlimited access to digital tools, financial professionals, and a knowledge center dedicated to financial wellness.

• Lyra enhanced mental health support available for employees and their dependents.

• Carrot fertility healthcare and family-forming benefits.

• Candidly student loan resources offered.

• Monthly stipend for remote work expenses.

• Quarterly lifestyle stipend provided.

• Team-building activities, including virtual social events and team offsites.

• Additional compensation elements such as bonuses, commissions, and equity may be included.

• Benefits focused on physical, financial, and mental wellness.