Senior Data Engineer, Databricks

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Establish and execute Data Engineering solutions that bridge business requirements with technological capabilities.

• Create and develop data architectures, incorporating Data Lake and Data Lakehouse frameworks.

• Build and oversee scalable data pipelines for both batch and streaming data processing.

• Evaluate and scrutinize data collection, organization, transformation, and delivery methodologies.

• Maintain data quality, governance, security, and performance standards.

• Collaborate with Data and Business teams to facilitate informed decision-making.

• Formulate data strategies that align with business goals.

• Provide mentorship to Data Engineering professionals.

⛳️ Requirements

• Extensive experience in Data Engineering and data architecture.

• Advanced proficiency in Azure Data Factory and Databricks.

• Familiarity with Apache Spark and distributed processing techniques.

• Proven experience in developing both batch and streaming data pipelines.

• Understanding of orchestration tools such as Airflow.

• Knowledge of Kafka and dbt.

• Proficiency in SQL and NoSQL databases, including PostgreSQL, MongoDB, and Cassandra.

• Awareness of cloud data platforms and services, including AWS and GCP.

• Experience with Data Lake and Data Lakehouse structures.

• Understanding of data governance, quality, and security practices.

🏝️ Benefits

• Competitive salary and comprehensive benefits package.

• Opportunities for professional development and growth.

• Flexible work environment and remote work options.

• Collaborative and inclusive company culture.