Senior SRE, Databricks

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Develop, maintain, and enhance Terraform modules for the provisioning of data infrastructure.

• Oversee state management, workspaces, and best practices for infrastructure versioning.

• Guarantee reproducibility, auditability, and traceability of infrastructure through code.

• Provision and manage networks (VPCs, subnets, firewall rules) adhering to security best practices.

• Configure IAM roles, policies, and service accounts based on the principle of least privilege.

• Manage Google Cloud Storage (GCS) as the data platform's storage layer.

• Ensure adherence to cloud security and governance policies.

• Provision and set up Databricks workspaces via Terraform/IaC.

• Manage clusters, jobs, notebooks, and permissions within the platform.

• Integrate Databricks with GCP infrastructure (service accounts, VPC, GCS, IAM).

• Construct and maintain CI/CD pipelines for infrastructure (GitHub Actions or similar).

• Implement GitOps practices: all infrastructure modifications through Pull Request with review and automated validation.

• Ensure secure and auditable deployment across multiple environments (dev/staging/prod).

• Implement secrets and credential management adhering to best practices (Secret Manager, Vault, etc.).

• Automate and standardize environments to ensure uniformity and eliminate manual configurations.

• Assist the data team with dependable, self-service infrastructure.

⛳️ Requirements

• Proficient in Infrastructure as Code (IaC): Terraform with modules, state management, and best practices.

• Familiarity with Google Cloud Platform (GCP): VPC, subnets, firewall rules, IAM (roles, policies, service accounts), GCS.

• Experience with Databricks: provisioning workspaces, managing clusters, jobs, and notebooks.

• Knowledge of version control using Git and GitOps practices.

• Experience with CI/CD pipelines (GitHub Actions, GitLab CI, Azure DevOps, or similar).

• Understanding of cloud security: IAM, secrets, access policies, and compliance.

• Ability to automate and standardize data environments.

• Experience with BigQuery: modeling, optimization, and integration with the data platform. [**DIFFERENTIAL**]

• Proficient in Apache Spark / PySpark for distributed processing. [**DIFFERENTIAL**]

• Familiarity with Delta Lake and Lakehouse architecture. [**DIFFERENTIAL**]

• Knowledge in Data Engineering and data pipelines (ETL/ELT). [**PLUS**]

• Experience with Kubernetes (GKE) for workload orchestration. [**PLUS**]

• Understanding of FinOps: cloud cost optimization, rightsizing, reservations. [**PLUS**]

• Familiarity with other IaC tools: Pulumi, Cloud Deployment Manager. [**PLUS**]

• Knowledge in observability: monitoring, logging, and alerting (Cloud Monitoring, Datadog, etc.). [**PLUS**]

🏝️ Benefits

• 🏥 Porto Seguro Health Insurance

• 🦷 Porto Seguro Dental Insurance

• 💰 Profit Sharing (PLR)

• 👶 Childcare Allowance

• 🍽️ Alelo Food and Meal Vouchers

• 💻 Home Office Allowance

• 📚 Partnerships with Educational Institutions

• 🚀 Support for Certifications, including Cloud

• 🎁 Livelo Points

• 🏋️‍♂️ TotalPass

• 🧘‍♂️ Mindself

• __Temporary project until December, with the possibility of extension into 2027.__