Data Engineering Lead

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design and develop reusable data platforms and systems with an emphasis on security.

• Establish architectural standards for data flows, ensuring both scalability and resilience.

• Oversee the creation of data ingestion, processing, and governance pipelines across cloud and on-premises settings.

• Suggest and enhance integration solutions between interdependent teams utilizing modern data tools.

• Apply best practices for observability, cost management, and security in distributed environments.

• Facilitate the distribution of development standards, code review methodologies, and release automation.

⛳️ Requirements

• Extensive experience with Databricks.

• Proficient programming capabilities in Python and strong SQL skills for managing and analyzing large datasets.

• Practical experience with Apache Airflow, Spark, S3/Data Lake, Snowflake, or BigQuery.

• Expertise in modeling and optimizing relational databases (PostgreSQL, MySQL) as well as NoSQL databases (DynamoDB).

• Familiarity with machine learning frameworks such as TensorFlow and PyTorch; orchestration using MLflow/Kubeflow.

• Skills in automation and infrastructure-as-code with Terraform or CloudFormation.

• Experience in implementing CI/CD pipelines for data and models, alongside managing Docker and Kubernetes environments for distributed processing.

• Setup of monitoring tools (Datadog) and visualization tools (QuickSight).

🏝️ Benefits

• Freedom to work from any location.

• Flexible working hours.

• Education assistance provided.

• Access to a dedicated internal career development tool.

• Participation in internal guilds and study/interest groups.

• Health insurance coverage.

• Dental insurance coverage.

• Pharmacy discount partnership available.

• 24/7 telemedicine services.

• Complimentary online therapy sessions.

• Wellhub wellness program.

• Extended maternity leave options.

• Extended paternity leave options.

• CAZ – Zuppers Support Center available.

• Meal and food vouchers provided.

• Life insurance coverage.

• Commuter allowance offered.

• Home office allowance included.

• Childcare assistance available.

• Phone plan allowance provided.

• Profit sharing opportunities.