Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Oversee infrastructure as code (IaC) utilizing Terraform to provision and configure AWS resources such as ECS, EMR, Lambda, Redshift, MSK, and S3, while leveraging the team's proprietary Terraform provider.

• Manage and enhance the Data Lake, incorporating Raw, Processed, and Refined zones, which includes deduplication, cataloging, and optimizing storage solutions (Parquet, Iceberg).

• Administer and monitor Kafka clusters, managing topic creation, connectors, ACLs, credentials, and tracking consumer lag for real-time streaming pipelines.

• Maintain and develop the Platform API, which provides features such as Data Loader, File Loader, Static Loader, Data Streaming, and Business Metrics for self-service access by other teams.

• Investigate and rectify pipeline incidents (failed DAGs, dataset desynchronization, data duplication, Redshift/Spark issues) by addressing the root causes.

• Engage in modernization efforts and integration with AI tools (Claude Code, MCP Servers).

• Contribute to the creation of technical documentation and uphold observability tools.

⛳️ Requirements

• Strong experience (3+ years) in Data Engineering or related fields.

• Proficient in Python for developing pipelines, automation scripts, and integrations.

• Practical experience with advanced SQL.

• Familiarity with Apache Airflow.

• Experience with AWS services including S3, Redshift, EMR (Spark), Lambda, ECS, and MSK (Kafka).

• Understanding of Apache Kafka, covering topics, producers/consumers, and connectors (Debezium, S3 Sink).

• Experience with Terraform or another Infrastructure as Code tool.

• Knowledge of Git and CI/CD workflows (Bitbucket Pipelines or similar).

• Understanding of Data Lake architectures.

• Effective communication skills and the ability to work independently in an agile team.

• Nice-to-haves:

• Experience with Apache Spark (PySpark, SparkSQL).

• Knowledge of C# / .NET.

• Familiarity with Debezium for Change Data Capture (CDC).

• Experience with modern table formats (Apache Iceberg, Hudi).

• Knowledge of Grafana for monitoring and operational dashboards.

• Experience with OpsGenie/JSM for incident and alert management.

• Familiarity with Redshift.

• Proficient technical English for reading documentation and communicating with LATAM teams.

🏝️ Benefits

• Bradesco National Health Plan — extended to dependents without beneficiary discount.

• Bradesco Dental Plan — optional.

• Flexible meal/food allowance (VR/VA) — maintained during vacation.

• Profit sharing (PLR).

• Wellhub.

• Birthday day off.

• Home office allowance.

• Commuter voucher (VT) as needed — legally permitted deductions.

• Life insurance.

• Free access to all our products — AppsClub, Discount Club, TrueCaller, BTFit, and Busuu.

• Access to internal training via digital platforms.

• Internal employee recognition program — Bemobucks.