Data Engineer – GCP, Databricks

This is a fully remote position, open to applicants in Brazil.

📋 Description

• A company dedicated to providing efficient and innovative services to its clients.

• A team of innovative professionals motivated by challenges.

• Technically oversee the advancement of the data platform.

• Mentor the team to ensure the delivery of reliable solutions in cloud environments.

⛳️ Requirements

• Spearhead the design and development of the data architecture (Data Lake, Data Warehouse, Lakehouse).

• Create, enhance, and optimize high-scale data pipelines (ETL/ELT).

• Act as the technical authority for the data engineering team.

• Work with distributed processing utilizing Databricks (Spark/PySpark).

• Establish best practices for development, testing, and deployment (DataOps).

• Ensure data governance, quality, security, and reliability.

• Optimize performance and expenses in GCP environments.

• Integrate various data sources (batch and streaming).

• Assist BI, Analytics, and Data Science teams in making data accessible.

• Engage in strategic decisions concerning data and technology.

🏝️ Benefits

• **Mandatory requirements**

• Specialist across various projects and technical team leadership.

• Minimum of 5 years' experience as a Data Engineer.

• Strong experience with Google Cloud Platform (BigQuery, Dataflow, Cloud Storage, Pub/Sub).

• Advanced expertise with Databricks and Apache Spark (PySpark).

• Proficiency in Python and SQL.

• Experience with data modeling (relational, dimensional, and Lakehouse).

• Familiarity with pipeline orchestration (Airflow/Cloud Composer or similar).

• Knowledge of code versioning and CI/CD practices.

• Experience with large-scale data environments.

• **Desired requirements**

• Experience with Delta Lake and Lakehouse architecture.

• Understanding of infrastructure as code (Terraform).

• Experience with Docker/Kubernetes.

• Knowledge of data streaming (Kafka, Pub/Sub).

• Relevant certifications (GCP Professional Data Engineer, Databricks).

• Experience with observability and monitoring tools.

• **Differentials**

• Experience leading technical teams or data projects.

• Strong communication skills with business stakeholders.

• Knowledge of data governance, LGPD, and information security.

• Experience with BI tools (Looker, Power BI, Tableau).

• Proficient in advanced English.