Senior Data Engineer

This is a fully remote position, open to applicants in Malta.

📋 Description

• Design and execute a robust, scalable data architecture that facilitates the collection, storage, and processing of significant volumes of igaming and customer data.

• Develop and sustain efficient ETL and ELT processes to guarantee smooth data ingestion from various sources into our data lake, DWH, and end consumers.

• Establish and uphold well-organized data models that ensure accuracy, integrity, and optimal performance for analytical applications, adhering to Kimball best practices.

• Collaborate with cross-functional teams to integrate data across systems, utilizing multiple integration methods such as API, Kafka, database extractions, web scraping, email, etc.

• Create and manage real-time data pipelines employing streaming technologies (e.g., Kafka) to support time-sensitive applications.

• Implement data governance policies and best practices, maintaining high standards for data quality, security, and regulatory compliance.

• Proactively monitor, troubleshoot, and enhance data pipelines, queries, and infrastructure to ensure speed, efficiency, and reliability.

• Assist in the development and expansion of production-grade orchestration workflows using Apache Airflow.

• Configure and maintain monitoring, alerting, and observability for data pipelines and systems utilizing tools like Elementary, Datadog, Prometheus, or Grafana.

• Create and uphold CI/CD pipelines for data projects using Bitbucket Pipelines or GitHub Actions, ensuring efficient deployment and testing procedures.

• Adhere to best practices in Git-based version control and collaborate effectively within a modern DevOps environment.

⛳️ Requirements

• Over 5 years of experience with Python, including at least 2 years working with Airflow.

• More than 5 years of experience with SQL in big data environments, ideally with at least 2 years of experience using dbt.

• At least 4 years of experience in cloud environments, preferably with 2+ years in both Snowflake and AWS.

• A minimum of 2 years' experience with data streaming platforms, preferably on Kafka.

• At least 2 years of hands-on experience with containerization technologies like Docker and Kubernetes.

• Over 3 years of managing CI/CD pipelines, ideally with Bitbucket Pipelines or GitHub Actions.

• A comprehensive understanding of data governance, data quality, and data lineage principles.

• Proficient in utilizing monitoring and alerting tools such as Datadog, Prometheus, or Grafana.

• Solid experience with version control systems (Git) and Infrastructure as Code (e.g., Terraform).

• Excellent communication and problem-solving skills, accompanied by a proactive and self-motivated attitude.

🏝️ Benefits

• Competitive salary aligned with skills and experience.

• Performance and bonus structure based on the achievement of set targets and individual performance.

• The chance to make a significant impact during a period of rapid growth.

• Full-time consultancy B2B contract with 25 days of paid time off (PTO).