Senior Data Engineer, Python, Databricks, Spark

This is a fully remote position, open to applicants in Portugal.

📋 Description

• Create and execute scalable data architectures.

• Build robust ETL pipelines.

• Assist in advanced analytics projects across various cloud platforms.

• Collaborate effectively with engineering leaders, stakeholders, and cross-functional teams.

• Participate in architectural decisions, engineering best practices, and initiatives for continuous improvement.

• Work with contemporary cloud-native data platforms that handle both real-time and batch processing workloads.

• Design comprehensive data pipelines from start to finish.

• Develop optimized OLAP systems.

• Facilitate scalable analytics platforms for intricate business domains.

⛳️ Requirements

• At least 5 years of professional experience in a Data Engineering role.

• Proficient hands-on experience with Python.

• Advanced SQL proficiency for large-scale data processing.

• Significant experience with Databricks and Apache Spark.

• Experience in designing and maintaining ETL pipelines using dbt.

• Familiarity with Apache Airflow for orchestration and workflow automation.

• Practical experience with cloud platforms such as AWS, GCP, or Azure.

• Experience with AWS S3, Google BigQuery, Google Cloud Storage, or Azure Databricks.

• Knowledge of Infrastructure as Code (IaC) principles using Terraform.

• Familiarity with event-driven or streaming technologies like Apache Kafka.

• Experience with Docker and Kubernetes.

• Strong understanding of dimensional data modeling techniques, including Kimball methodology.

• Experience in designing OLAP systems, Star Schemas, and Semantic Models.

• Understanding of data architecture methodologies such as Inmon and Data Vault.

• Experience with CI/CD workflows and Git version control systems.

• Excellent communication and collaboration skills.

• Ability to engage effectively with both technical and non-technical stakeholders.

• Proficiency in English.

🏝️ Benefits

• Competitive salary.

• Opportunities for professional development and growth.

• Flexible work arrangements.

• Access to cutting-edge technologies.