Staff Data Engineer

This is a fully remote position, open to applicants in California, +19 more states.

📋 Description

• Take ownership of the creation, maintenance, and enhancement of pipelines that facilitate data ingestion from operational databases and third-party tools into a data lake/warehouse.

• Design highly efficient ingestion patterns that adeptly manage evolving data schemas and high-volume, multi-source data streams.

• Enhance pipeline performance to guarantee maximum uptime, high throughput, and cost-effective compute resource usage.

• Leverage dbt to convert raw data within the data warehouse into structured, production-ready schemas.

• Develop templated SQL and Jinja code to uphold macro-driven, modular, and DRY (Don't Repeat Yourself) development methodologies.

• Conduct thorough data quality assessments by implementing native dbt tests.

• Establish monitoring and alerting frameworks for both ingestion routines and dbt builds.

• Monitor pipeline health metrics to evaluate and report on overall data freshness and platform reliability.

• Create data applications that engage with the data warehouse to empower decentralized self-service analytics.

• Develop internal tools and libraries to support analytics and machine learning initiatives.

⛳️ Requirements

• Demonstrated production experience in building, scaling, and sustaining robust ingestion pipelines utilizing APIs, CDC (Change Data Capture), and orchestrators.

• Advanced expertise in dbt, encompassing production deployments, package management, and the creation of custom Jinja macros.

• Extensive hands-on experience in manipulating and managing data within cloud data systems (e.g., Snowflake, BigQuery, Databricks).

• Strong proficiency in Python or comparable back-end languages for developing operational applications and custom pipeline tools.

• Familiarity with setting up alerting infrastructure and experience with version control (Git) and CI/CD pipelines.

🏝️ Benefits

• Unlimited PTO and paid holidays

• Medical/Dental/Vision coverage provided to all full-time employees

• 401(k) plan with matching contributions.