Staff Software Engineer – Semantic Foundation

This is a fully remote position, open to applicants in California, +5 more states.

📋 Description

• Design, implement, scale, and sustain highly available Apache Airflow clusters utilizing Helm, Kubernetes, and Terraform.

• Lead DevOps initiatives using Terraform, Helm Charts, and ArgoCD.

• Automate platform deployments, oversee self-hosted GitHub runners, and uphold GitOps workflows.

• Architect and manage CI/CD pipelines through GitHub Actions for data pipelines, infrastructure elements, and DAGs.

• Provision and manage Kubernetes EKS/AKS clusters, Docker containers, and AWS/Azure cloud infrastructure.

• Develop and maintain monitoring, logging, and alerting systems using Grafana, Prometheus, Loki, and centralized log management solutions.

• Construct scalable data infrastructure for Apache Spark, AWS EMR, Snowflake, Azure Synapse, Apache Kafka, and dbt.

• Deploy and maintain data discovery and metadata tools such as DataHub.

• Monitor, audit, and optimize AWS and Azure compute and storage expenditures.

• Create internal tools, CLI utilities, and dynamic workflow templates.

• Take ownership of data platform modules from architecture through deployment, production operations, and incident management.

• Establish and uphold engineering standards related to code quality, design, testing, data lineage, security, access controls, IAM, and schema management.

• Collaborate with data leads, product managers, and business stakeholders on a 1–2 year data platform roadmap.

• Mentor junior and mid-level data platform engineers.

⛳️ Requirements

• Over 6 years of practical experience in Data Platform Engineering, DevOps, or Site Reliability Engineering within large-scale data environments.

• Extensive production experience in managing, tuning, dynamically scaling, and troubleshooting Apache Airflow infrastructure.

• Expert proficiency in Kubernetes, Helm, Terraform, Docker, ArgoCD, and GitHub Actions, including custom runner setups.

• Demonstrated experience in configuring production alerting, metrics collection, and log aggregation using Grafana, Prometheus, and Loki.

• In-depth operational and configuration expertise with Spark, AWS EMR, Snowflake, Azure Synapse, dbt, and Apache Kafka.

• Hands-on experience with AWS and/or Azure, showcasing a history of optimizing cloud costs.

• Experience in deploying or managing data cataloging tools like DataHub, Amundsen, or similar metadata management platforms.

• Strong programming abilities in Python, Bash, Go, or SQL.

• Capable of transforming complex architectural requirements into production-grade deployments.

• Committed to automating manual operational tasks with code, automated tests, automated CI/CD checks, and resilient self-healing infrastructure.

• Must reside within 30 miles of Portland, ME; Boston, MA; Chicago, IL; Dallas, TX; San Francisco Bay Area, CA; or Seattle, WA.

🏝️ Benefits

• Health, dental, and vision insurance.

• Retirement savings plan.

• Paid time off.

• Health savings account.

• Flexible spending accounts.

• Life insurance.

• Disability insurance.

• Tuition reimbursement.

• Eligibility for quarterly or annual bonuses for non-sales roles.