Senior Data Platform Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Modify or create resources within cloud environments such as AWS and Azure;

• Alter or develop new resources in Databricks;

• Adjust or establish new application resources in Kubernetes;

• Suggest and execute architectural enhancements;

• Enforce controls related to LGPD (Brazil's General Data Protection Law) within the data lake;

• Develop dashboards and perform metadata analyses in Databricks for monitoring and compliance;

• Carry out audits with the governance team to ensure the data lake aligns with the company’s security policies;

• Monitor and assess costs to pinpoint bottlenecks and identify opportunities for cost reduction;

• Collaborate with data analytics and AI teams to enhance job performance and resource utilization;

• Work alongside other platform engineers to support skill development;

• Act as a technical reference for platform-related inquiries from data, business, and product stakeholders.

⛳️ Requirements

• Bachelor’s degree or an equivalent qualification;

• In-depth knowledge of relational and non-relational databases, including patterns, modeling, and use cases;

• Proficiency in programming languages for maintaining data pipelines (Python, shell/bash) and file handling formats such as JSON, Parquet, and CSV;

• Experience in utilizing APIs, along with data cleaning, processing, deduplication, enrichment, and normalization;

• Familiarity with messaging infrastructure utilizing Kafka and Debezium;

• Experience with Kubernetes cluster infrastructure like EKS and AKS;

• Strong SQL skills along with basic knowledge of DBT;

• Background in cloud computing projects, particularly with AWS and/or Azure;

• Familiarity with Gitflow and version control systems;

• Experience utilizing Infrastructure as Code (Terraform);

• Knowledge in distributed processing (Databricks/Spark);

• Experience with Docker;

• Familiarity with orchestration tools such as Apache Airflow and Databricks Workflows;

• Understanding of data architecture;

• Experience in developing CI/CD pipelines;

• Proficiency in observability tools (Grafana/CloudWatch).

🏝️ Benefits

• Meal allowance / meal voucher;

• Flexible working hours and arrangements (for remote positions);

• Transportation allowance (for hybrid or on-site roles);

• Profit-sharing (PLR);

• Multi-benefits: Flexible benefit through Flash Card for personal use;

• Gympass / Wellhub;

• Psicologia Viva (an online platform for psychology and nutrition consultations);

• Health and dental insurance;

• Life insurance;

• Extended parental leave (up to 6 months for mothers and 20 days for fathers);

• Rede D'Or: support and essential information for maternal and child health, including a network of accredited nurses;

• Partnership with your regional SESC (offering varied programming in education, health, culture, leisure, and assistance);

• Birthday day off (one day off to celebrate your birthday or during your birthday month);

• Access to a platform with a variety of courses to enhance your knowledge (UCA);

• Language academy (AIA);

• Leadership development program;

• Mentoring for Women at Afya (MMA);

• Discounts on undergraduate and graduate courses at Afya educational units.