Data Engineer – Mid-level

📋 Description

• Pipeline Development: Create, construct, and sustain data pipelines (ETL/ELT) utilizing Pentaho (PDI) for legacy systems and AWS Glue/Airflow for contemporary cloud architectures.

• Orchestration and Automation: Oversee the execution of jobs and operational routines using Rundeck and Airflow, ensuring data observability.

• Data Modeling: Develop intricate SQL queries to convert raw data from various processors into analytical perspectives for the business.

• Coding and Versioning: Employ Python for automation and tailored transformations, managing the complete code lifecycle through Git.

• Multi-partner Integration: Manage the ingestion of data from multiple partners (RPE, Dock, etc.), standardizing information from diverse sources.

⛳️ Requirements

• Pentaho Data Integration (PDI): Advanced expertise in handling complex transformations and jobs.

• SQL (native): Strong ability to manipulate substantial volumes of data and enhance query performance.

• Python: Focus on data analysis and API integration.

• Cloud & Modern Stack: Familiarity with AWS Glue and orchestration using Airflow.

• Operations: Understanding of Rundeck for task management and Git for version control (Gitflow).

🏝️ Benefits

• Not specified