Senior Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design, develop, and maintain scalable and resilient data architectures and pipelines.

• Engage in data integration across various sources (both internal and external), ensuring consistency and traceability.

• Develop and enhance ETL/ELT processes on cloud data platforms.

• Ensure the quality, integrity, and governance of the provided data.

• Assist business areas and analytics teams by delivering reliable data for decision-making.

• Construct and sustain data models and layers that facilitate visualization and analytics solutions.

• Collaborate with engineering, analytics, product, and business teams.

• Suggest ongoing improvements in the performance, cost, and reliability of the data ecosystem.

⛳️ Requirements

• Demonstrated experience in data architecture and integration.

• Strong understanding of ETL/ELT tools and cloud data platforms.

• Familiarity with visualization tools (such as Power BI, Tableau, or similar); experience with Databricks is an advantage.

• Experience in data governance and ensuring data quality.

• Extensive experience with the Azure ecosystem (including Azure Data Lake Storage, Azure Data Factory, Azure Key Vault, and Synapse).

• Practical experience with Databricks, including Delta Lake and Medallion architecture patterns (Bronze, Silver, Gold).

• Proficiency in Python (PySpark) and SQL (Spark SQL) for processing large data volumes in a distributed environment.

• Experience in data modeling (Data Lake, Lakehouse, and Data Warehouse).

• Understanding of DataOps practices, versioning with Git, and CI/CD pipelines (Azure DevOps or similar).

• Ability to identify and address issues related to data integrity, quality, and performance.

• Strong communication skills and the ability to interact with multidisciplinary teams (engineering, analytics, product, and business).

• Intermediate proficiency in English.

• Preferred qualifications include:

• Experience in the financial sector or payments.

• Familiarity with Unity Catalog (data governance and lineage) or equivalent tools (e.g., Purview).

• Experience with real-time processing (Structured Streaming, Event Hubs, or Kafka).

• Knowledge of Infrastructure as Code (Terraform or similar).

• Certifications in Databricks and/or Azure Data Engineer Associate.

🏝️ Benefits

• Competitive compensation based on experience.

• Opportunities for career advancement and involvement in strategic projects.

• A dynamic and challenging work environment.

• The chance to work at a rapidly growing company.