Senior Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Develop and execute data ingestion, transformation, and orchestration pipelines on Google Cloud.

• Maintain the quality, reliability, and scalability of both batch and streaming data flows.

• Optimize storage and processing expenses in BigQuery, Dataflow, Dataproc, and Pub/Sub.

• Work collaboratively with data scientists and analysts to provide consumption-ready data.

⛳️ Requirements

• Experience in data ingestion using Dataflow (Apache Beam) for scalable ETL/ELT pipelines, including extraction from external sources like Oracle on OCI and on-premises SQL Server.

• Skill in real-time ingestion with Pub/Sub and batch ingestion utilizing Storage Transfer, including sources such as AWS S3 buckets.

• Ability to integrate with Oracle, SQL Server, and MySQL through JDBC, Dataflow templates, or custom extraction methods.

• Proficiency in BigQuery, covering data modeling, query optimization, partitioning, and clustering.

• Experience using Cloud Storage as a layered data lake (raw, bronze, silver, gold).

• Familiarity with Dataform or dbt for SQL orchestration in BigQuery, along with best practices in dimensional modeling.

• Knowledge of batch and streaming processing with Beam and streaming SQL.

• Experience in orchestration using Cloud Composer (Airflow) or Workflows.

• Integration capabilities with Cloud Functions and Cloud Run for serverless tasks.

• Proficiency in advanced SQL (BigQuery SQL), Python (for Dataflow, automation, ETL), and Git with CI/CD practices.

• Familiarity with Dataproc, Bigtable, and Firestore is a plus.

• Knowledge of Dataplex and Data Catalog for data documentation and cataloging is an advantage.

• Understanding of healthcare systems, including PEP/EMR/EHR, RIS, LIS, and scheduling and referral management systems, is desirable.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health and wellness benefits.

• Opportunities for professional development and career advancement.

• Flexible work arrangements and a supportive work environment.