Senior Data Engineer

This is a fully remote position, open to applicants anywhere in the world.

📋 Description

• Collaborate with product and engineering teams to gather requirements and translate them into technical architecture.

• Assess and suggest data tools, frameworks, and infrastructure options.

• Participate in roadmap discussions and make build-vs-buy evaluations.

• Design, implement, and maintain scalable AWS data infrastructure.

• Take ownership of data lake and warehouse architecture, focusing on partitioning, storage optimization, and data lifecycle management.

• Create and manage production-grade Apache Airflow DAGs for ingestion, transformation, and export workflows.

• Ensure observability through monitoring, alerting, and incident management.

• Develop and sustain dbt pipelines with data quality checks and structured data modeling.

• Design and maintain database schemas for multi-state, multi-tenant program data.

• Write and optimize SQL queries across PostgreSQL, Redshift, and Athena.

• Create reusable data models, utilities, and shared Python packages.

• Architect infrastructure for large-scale time-series and event-based data.

• Implement ingestion, storage, and retrieval patterns for high-frequency temporal datasets.

• Work with vector databases to enable AI-powered features and semantic search.

• Integrate LLM workflows into ELT pipelines utilizing AWS Bedrock, LangChain, and related frameworks.

• Develop production AI-assisted data comparison, validation, and enrichment pipelines.

• Utilize MLflow and deploy production-level ML pipelines.

• Stay updated on AI/ML tooling advancements and introduce relevant innovations to the team.

• Design automated data QA systems to ensure dataset quality, completeness, and consistency.

• Implement cleansing and reconciliation routines for multi-source ingestion.

• Proactively monitor pipelines and resolve issues to prevent downstream impacts.

• Mentor junior and mid-level data engineers through code reviews, pair programming, and architectural guidance.

• Establish standards for code quality, testing, and documentation.

• Promote a culture of ownership, curiosity, and continuous improvement within the data team.

⛳️ Requirements

• Over 5 years of data engineering experience with full ownership and operational support of production systems end-to-end.

• Ability to assess technical trade-offs and make pragmatic architecture decisions that balance cost and performance.

• Strong competency in SQL, including PostgreSQL and Athena/Presto.

• Proficient in Python and production-quality coding practices.

• Extensive hands-on experience with AWS data services, including S3, Athena, RDS, ECS, Lambda, IAM, and CloudWatch.

• Practical experience with Apache Airflow or similar orchestration tools.

• Familiarity with large-scale time-series, event-based, or streaming data systems.

• Experience with version control systems such as GitHub, Subversion, GitLab, or Mercurial.

• Knowledge of vector databases and LLM integration patterns like LangChain and AWS Bedrock.

• Experience in creating data quality frameworks or automated validation systems.

• Proven ability to mentor engineers and contribute positively to team culture.

• Excellent communication skills for articulating complex infrastructure decisions to non-technical stakeholders.

• Comfortable navigating ambiguity and making decisions based on first principles.

• Willingness to work across multiple time zones.

• Experience managing data for SaaS platforms is advantageous.

• Background in large-scale spatiotemporal data pipelines, including PostGIS, spatial indexing, and tiling, is a plus.

🏝️ Benefits

• Competitive (and continuously expanding) benefits for employees and their dependents.

• Opportunities for learning and growth – encompassing all aspects of startups.

• Flexibility to work from any location, provided there is reliable internet access.

• Optional and encouraged retreats 2–3 times a year.

• Chance to help shape the benefits program.

• Meaningful equity upside.