Senior Software Engineer, Data Platform

This is a fully remote position, open to applicants in California.

📋 Description

• Design and construct high-throughput data pipelines from the ground up.

• Create pipelines for the processing, alignment, transformation, and calibration of intricate datasets.

• Develop frameworks for backfilling and reprocessing historical data.

• Establish dataset lineage, versioning, and reproducibility protocols.

• Create APIs and tools for dataset discovery and accessibility.

• Develop metrics for data quality, monitoring, and alerting systems.

• Optimize storage solutions, compression techniques, partitioning, and data access methodologies.

• Construct internal tools and workflows for machine learning and data labeling teams.

• Collaborate closely with machine learning engineers to comprehend and fulfill their data needs.

⛳️ Requirements

• 5–8 years of experience in software or data engineering.

• Demonstrated capability to design a data platform or high-throughput pipeline from scratch.

• Proficient in Python and familiar with data tools such as PyArrow, Polars, Pandas, NumPy, or SciPy.

• Experience with AWS or another major cloud platform, including the deployment of production data pipelines.

• Familiarity with a service-oriented language such as Go, Rust, or TypeScript.

• Solid understanding of data lineage, versioning, reproducibility, and data quality standards.

• Experience in building backfilling and reprocessing systems.

• High sense of ownership and the capability to work independently in a startup setting.

• U.S. citizenship is required.

🏝️ Benefits

• 0.25%–0.5% equity