Engenheiro de Dados Sênior

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Engage in the maintenance and enhancement of multi-stage ETL pipelines across various data domains;

• Implement data transformations, including conversions, outlier filtering, gap filling, smoothing, and interpolation;

• Diagnose and resolve data quality issues in production pipelines;

• Design and maintain field mapping configurations based on YAML for new data sources;

• Query and load data using Cloud SQL and BigQuery;

• Construct and maintain endpoints using FastAPI while adhering to clean architecture principles;

• Develop unit and integration tests utilizing pytest;

• Collaborate on code reviews and maintain CI/CD pipelines in Azure DevOps;

• Work directly with raw data and production pipelines, ensuring efficiency and reliability;

• Translate business rules into efficient transformations using pandas;

⛳️ Requirements

• Advanced English for daily communication with international teams;

• Solid experience with Python (version 3.10 or higher);

• Proficiency with dataclasses, type hints, and abstract base classes;

• Experience with pandas and NumPy for large-scale data processing;

• Experience with scipy for optimization and curve fitting;

• Experience with SQLAlchemy (Core and ORM);

• Background in building asynchronous APIs using FastAPI and asyncio;

• Experience with Cloud SQL and IAM authentication in GCP;

• Experience with BigQuery for data querying and loading;

• Experience with Cloud Storage for file manipulation;

• Knowledge of service accounts and access control (IAM);

• Experience in designing and debugging multi-stage data pipelines;

• Strong understanding of sequential data processing or time series;

• Experience with data validation, deduplication, and imputation;

• Experience with configuration-driven pipelines (YAML);

• Experience with Docker and Docker Compose;

• Experience with CI/CD pipelines (preferably Azure DevOps);

• Experience managing dependencies using Poetry;

• Basic knowledge of Redis as a caching layer;

• Ability to communicate technical decisions clearly and collaborate effectively with multidisciplinary teams;

🏝️ Benefits

• Multi-benefit card – you choose how and where to use it.

• Scholarships for Undergraduate, Graduate, MBA, and Language courses.

• Certification incentive programs.

• Flexible working hours.

• Competitive salaries.

• Annual performance evaluation with a structured career plan.

• Opportunities for international career advancement.

• Wellhub and TotalPass.

• Private pension plan.

• Childcare assistance.

• Medical assistance.

• Dental assistance.

• Life insurance.