Principal Data Engineer

This is a fully remote position, open to applicants in California.

📋 Description

• Design, construct, and expand data pipelines that integrate data into the platform, preparing it for analytical purposes.

• Facilitate the movement of data through raw/source, staging, and curated layers, ensuring appropriate transformation, validation, and testing.

• Develop and sustain Azure Data Factory pipelines to orchestrate ingestion and transformation across source, curated, and serving layers.

• Create ingestion patterns for REST/SOAP APIs, CSV and fixed-width files, JSON, and XML formats.

• Establish robust API extraction logic that accommodates pagination, rate limiting, incremental/delta pulls, and schema drift.

• Process and flatten semi-structured JSON payloads into queryable relational formats within Snowflake.

• Manage file-based ingestion at scale, including handling SFTP/blob drops, file validation, schema enforcement, and reprocessing/backfill logic.

• Contribute to the CI/CD of ADF pipelines across development, UAT, and production environments utilizing Azure DevOps/GitHub Actions.

• Set technical benchmarks for pipeline design, code quality, and testing protocols.

• Provide mentorship to fellow data engineers.

• Collaborate with data engineers, data modelers, analysts, product teams, and engineering teams on data infrastructure priorities.

⛳️ Requirements

• Practical experience with Azure Data Factory (ADF) — including pipelines, data flows, linked services, integration runtimes, and triggers.

• Background in extracting and normalizing raw data from REST APIs, flat files, JSON/XML, and SFTP/blob drops.

• Experience managing pagination, authentication (API keys, OAuth), and incremental synchronization patterns.

• Proficiency in parsing and modeling semi-structured/nested JSON data within a cloud data warehouse, such as Snowflake VARIANT/FLATTEN.

• Over 8 years of experience in data engineering or a related field, including experience at a senior or technical lead level.

• Strong expertise in SQL.

• Proficiency in at least one data engineering programming language: Python, Scala, or Java.

• Exceptional communication skills and cross-functional experience with analytics, product, and engineering teams.

• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience is preferred.

• Familiarity with dbt or similar transformation frameworks.

• Knowledge of infrastructure-as-code practices, including Terraform or ARM/Bicep.

• Familiarity with general CI/CD practices.

• Exposure to real-time analytics or streaming technologies.

🏝️ Benefits

• Health Insurance

• Savings and Retirement Plan

• Employee Assistance Program

• Generous Vuori Discount & Industry Perks

• Paid Time Off

• Wellness & Fitness benefits

• Bonus eligibility

• Remote work arrangement