Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Data Pipeline Creation: Develop and manage ETL (Extract, Transform, Load) processes to transfer data from various sources into data warehouses and data lakes.

• Infrastructure and Database Management: Design and deploy scalable databases (both relational and NoSQL), ensuring optimal security, performance, and data integrity.

• Big Data Management and Distributed Processing: Oversee large datasets utilizing distributed processing tools, crucial for organizations with significant data demands.

• Cloud Solutions and Workflow Automation: Implement cloud technologies (AWS, Azure, GCP) and automate processes using Infrastructure as Code (IaC) and DevOps methodologies.

• Collaboration with Analytics Teams: Partner with analytics departments to comprehend business requirements, ensuring data is primed for dashboards and AI models.

⛳️ Requirements

• Extensive expertise in Spark, Delta Tables, Python, SQL, and Git, with solid experience in Databricks.

• Databricks Professional certification is required.

• Familiarity with DABs, DLT, Spark Structured Streaming, Delta Sharing, and MLflow.

• Proven experience in configuring agents using LangChain or similar frameworks in Python.

• Background in the financial industry is preferred.

🏝️ Benefits

• Meal and food allowances.

• Health insurance coverage for you and your dependents.

• Dental insurance for you and your dependents.

• Two complimentary therapy sessions per month with a psychologist.

• Nutritionist consultations covered by Credsystem.

• Gympass access for your health and wellness.

• Weekly well-being sessions including massage therapy to promote relaxation.

• Life insurance coverage.

• Variable compensation based on role and achievement of objectives.

• Discounts on university tuition to support your career advancement.

• Partnership with a language school to enhance your language skills.

• Day off for your birthday.

• Payroll-deductible loans available.

• Options for transportation vouchers, shuttle service, or parking assistance.

• Childcare support.

• Sesc benefits.