Senior Data Engineer – Azure Cloud

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design, maintain, and enhance scalable data pipelines.

• Utilize distributed processing techniques with Apache Spark.

• Manage workflows and data routines using Apache Airflow.

• Create solutions in Python adhering to engineering best practices.

• Operate within cloud platforms, particularly Microsoft Azure.

• Establish and uphold deployment pipelines employing CI/CD methodologies.

• Work with configuration and automation files in YAML/YML format.

• Implement Docker for the containerization of applications and services.

• Develop, test, and document data transformation models utilizing dbt.

• Collaborate with engineering, analytics, and business teams to guarantee solution quality, reliability, and performance.

• Assist in making technical decisions regarding architecture, performance, observability, and data governance.

⛳️ Requirements

• Proficiency in English is a mandatory requirement.

• Experience with Azure and associated data services.

• Strong knowledge of Python programming.

• Familiarity with Apache Spark.

• Experience with Apache Airflow.

• Understanding of CI/CD practices.

• Expertise in development and configuration with YAML/YML.

• Knowledge of Docker technology.

• Experience with dbt for data transformation and modeling.

• Awareness of code versioning, testing, documentation, and development best practices.

• Experience in consulting projects is a plus.

• Knowledge of lakehouse architecture, data lakes, or data warehouses is a plus.

• Familiarity with Databricks, Azure Data Factory, Synapse, or Fabric is a plus.

• Understanding of observability, monitoring, and data quality is a plus.

• Experience with critical production environments and high-volume data pipelines is a plus.

🏝️ Benefits

• Position also open to candidates with disabilities.