Data Lead

This is a fully remote position, open to applicants in Latin America.

📋 Description

• Drive the advancement of a production data platform centered around dbt, Apache Airflow, Amazon Redshift, Python, and AWS.

• Assume responsibility for the platform's architecture, dependability, and data integrity.

• Design, operate, maintain, and enhance production data platforms and contemporary data warehouse architectures.

• Create and uphold software solutions across a diverse technology stack.

• Navigate, troubleshoot, and enhance intricate dbt projects with numerous models and thousands of automated tests.

• Design and oversee Apache Airflow DAGs, operators, task groups, sensors, variables, connections, and SLAs.

• Optimize Amazon Redshift schemas, distribution and sorting strategies, query performance, and warehouse expenses.

• Develop and sustain ingestion processes.

• Manage data infrastructure utilizing Terraform.

• Offer technical leadership, documentation, architectural insights, and knowledge transfer.

• Maintain and evolve a complex, mission-critical data ecosystem.

⛳️ Requirements

• Extensive experience in designing, operating, and evolving production data platforms and modern data warehouse architectures.

• Deep hands-on experience with dbt in large-scale production settings, encompassing project structure, model layering, macros, seeds, tests, package management, and dbt Cloud job orchestration.

• Proven capability to navigate, troubleshoot, and enhance complex dbt projects with numerous models and thousands of automated tests.

• Strong practical experience with Apache Airflow, including DAG design, operators, task groups, sensors, variables, connections, and SLA configuration.

• In-depth hands-on experience with Amazon Redshift, including schema design, distribution and sorting strategies, query performance optimization, and warehouse cost management.

• Familiarity with Redshift Serverless and an understanding of RPU usage and consumption-driven cost behavior.

• Strong skills in dimensional and data warehouse modeling, including Slowly Changing Dimensions (SCD Type 2).

• Advanced SQL proficiency, with the capability to comprehend, troubleshoot, and enhance complex existing data models and queries.

• Strong Python skills applied to data engineering, ingestion, orchestration, and data processing.

• Experience managing data infrastructure through Terraform.

• Solid understanding of data quality and automated testing strategies across data pipelines, transformations, and warehouse models.

• Ability to construct and maintain ingestion processes rather than solely relying on managed ELT platforms.

• Strong ownership of technical documentation, architectural knowledge, and knowledge transfer processes, with the ability to absorb and preserve critical platform knowledge progressively.

• Experience working within established architectures, with the discernment to understand existing systems before suggesting significant changes.

• Experience with entity resolution, record matching, master data management, data lineage, or auditability is highly valued.

• Experience with AWS Managed Workflows for Apache Airflow (MWAA) is highly valued.

• Experience managing costs and performance in consumption-based data warehouse environments is valued.

• Experience in regulated, compliance-driven, or high-accountability data environments is valued.

• Strong technical leadership, ownership, and decision-making skills, including a clear perspective on data architecture, pipeline reliability, testing, and maintainability.

• Professional English communication skills and experience collaborating directly with client stakeholders.

🏝️ Benefits

• 100% Remote.

• Above-average salaries in USD.

• Profit Sharing Policy.

• Unlimited PTOs after your first year (20 PTOs in the first year).

• Equipment provided for your setup.

• Excellent work/life balance.

• Honest, straightforward, and transparent culture.

• US-based clients and international teams.