Senior Data Platform Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Provide support, administration, and advancement of the data platform within a Kubernetes environment utilizing Google Kubernetes Engine (GKE).

• Create and improve Python applications, encompassing web portals, internal APIs, and data APIs.

• Automate processes related to customer, user, license, and access management.

• Administer and upgrade platform components using Helm and GitOps methodologies with Argo CD.

• Enhance the platform architecture, including version upgrades, container image creation and maintenance, and infrastructure as code.

• Develop and sustain CI/CD pipelines for both applications and platform components.

• Implement and manage identity and access control systems utilizing OAuth 2.0, OpenID Connect, and Keycloak or similar solutions.

• Establish backup, restoration, and recovery protocols for PostgreSQL and persistent volumes in Kubernetes.

• Collaborate with Google Cloud Platform services, including IAM, Secret Manager, Cloud Build, Cloud Functions, and Cloud SQL.

• Create and maintain integrations and automations using Python alongside the platform’s data services.

• Implement monitoring, observability, and alerting solutions to guarantee service availability and reliability.

• Identify opportunities for cost optimization and resource utilization within Google Cloud.

• Develop Artificial Intelligence agents to enhance customer service tasks, incident diagnosis, and backup validation.

⛳️ Requirements

• Extensive senior-level expertise in Platform Engineering, Data Engineering, DevOps, or Site Reliability Engineering (SRE).

• Strong practical experience with Kubernetes in production settings.

• Proficient with Google Cloud Platform (GCP), specifically IAM, Secret Manager, Cloud Build, Cloud Functions, and Cloud SQL.

• Familiarity with Helm and GitOps methodologies, preferably with Argo CD.

• Proficient in Python for application development, integrations, and automation processes.

• Understanding of SQL and BigQuery.

• Experience with Apache Airflow, ideally using Cloud Composer.

• Background in developing Python applications, including REST APIs and frameworks such as FastAPI or Flask.

• Knowledge of automated testing, code version control, and CI/CD pipelines.

• Understanding of authentication and authorization processes using OAuth 2.0 and OpenID Connect.

• Experience managing Keycloak or equivalent identity and access management solutions.

• Familiarity with backup and restoration methods for PostgreSQL and persistent volumes in Kubernetes.

• Experience with Apache Superset.

• Knowledge of or experience with Streamlit.

• Experience with Terraform and infrastructure-as-code methodologies.

• Understanding of security measures for applications accessible via the internet.

• Background in developing and utilizing Artificial Intelligence agents.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Flexible work hours and remote work options.

• Opportunities for professional development and continued education.

• Collaborative and inclusive work environment.

• Access to the latest tools and technologies.