Senior Data Engineer – Snowflake, dbt, AWS

This is a fully remote position, open to applicants in Portugal.

📋 Description

• Develop and manage batch and streaming ingestion pipelines from various sources including databases, event streams, and third-party APIs.

• Take ownership of the storage architecture encompassing lake, lakehouse, and warehouse, with a focus on partitioning, cost efficiency, and query performance.

• Ensure pipelines are observable and restartable through effective orchestration, automated testing, alerting, and seamless recovery.

• Implement software engineering best practices in data management, including infrastructure as code, CI/CD, code reviews, and version control.

• Establish data contracts with producing teams to mitigate schema drift.

• Oversee access, retention, and data lineage.

• Create documentation that endures beyond the engagement period.

• Collaborate directly with senior leadership and client teams on production data systems.

• Construct and manage pipelines and platforms that facilitate the transfer of data from source systems to the warehouse for a prominent retail client.

⛳️ Requirements

• Over 5 years of experience as a Data Engineer, with hands-on experience building and managing production pipelines.

• Proficient in advanced, production-grade SQL and Python 3.

• Extensive background in ETL/ELT processes and CI/CD for data workloads.

• Practical experience with dbt.

• Familiarity with orchestration tools such as Apache Airflow, Azure Data Factory, or Dagster.

• Extensive experience with Snowflake and Terraform.

• Strong grasp of data modeling methodologies including Kimball, Inmon, and Data Vault.

• Production experience with AWS.

• Utilization of AI tools in daily tasks, with a disciplined review process for generated outputs prior to deployment.

• Proficient in English (both spoken and written), comfortable engaging directly with client teams.

• Experience with an additional cloud provider (GCP or Azure) is preferred.

• Familiarity with Spark/PySpark at scale and Databricks is preferred.

• Knowledge of streaming and pub/sub brokers such as Kafka, Kinesis, Pub/Sub, or MQTT is preferred.

• Experience with open table and file formats including Parquet, Apache Iceberg, and Delta Lake is preferred.

• Familiarity with Docker, Kubernetes, and GitHub Actions is preferred.

• Experience with data quality frameworks in production such as Great Expectations, Soda, or Monte Carlo is preferred.

• Knowledge of AI data infrastructure, including embedding pipelines, vector stores, and feature stores is preferred.

• Familiarity with NoSQL databases (e.g., MongoDB) and marketing data sources (Google Ads, Meta, CRM APIs) is preferred.

• A cost engineering mindset, focusing on reducing pipeline costs and quantifying savings is preferred.

• Experience working in regulated environments (GDPR, access control, auditable lineage) is preferred.

🏝️ Benefits

• 15th month salary.

• Health insurance covering your family.

• Birthday off.

• Mobility program for digital nomads.

• Real work-life balance.