Senior Data Engineer, Databricks

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Develop and maintain data ingestion, transformation, and availability pipelines.

• Design and structure data in Data Lake and Data Lakehouse environments.

• Create ETL/ELT processes using Databricks, Python, and Azure Data Factory.

• Integrate data from APIs, databases, as well as structured and unstructured files.

• Engage in projects involving SAP Datasphere and Azure Data Lake Storage (ADLS).

• Build solutions for data processing with an emphasis on performance, quality, and governance.

• Participate in Generative AI initiatives, such as structuring datasets for RAG and document processing.

• Collaborate with multidisciplinary teams to enhance data solutions.

⛳️ Requirements

• Bachelor's degree in Information Technology or a postgraduate degree in IT, with a minimum of 360 hours recognized by MEC.

• Advanced experience with Databricks.

• Strong knowledge in data modeling.

• Proficient in Python and SQL.

• Familiarity with Azure Data Factory and Azure Data Lake Storage (ADLS).

• Experience with SAP Datasphere.

• Background in relational databases, such as SQL Server and Oracle.

• Proven experience in building and maintaining scalable and reliable data pipelines.

• Preferred: knowledge of Apache Spark, Airflow, Kafka, or dbt.

• Preferred: experience with Data Lake and Data Lakehouse architectures.

• Preferred: experience with API integrations.

• Preferred: experience in Generative AI projects, RAG, and document processing.

• Preferred: knowledge of other cloud platforms and modern data ecosystems.

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Opportunities for professional development and continuous learning.

• Flexible working hours and remote work options.

• Comprehensive health and wellness benefits.

• Collaborative and inclusive work environment.