Senior Data Engineer

This is a fully remote position, open to applicants in South Africa.

📋 Description

• Design and develop robust, scalable, and high-performance ETL/ELT data pipelines.

• Create and implement solutions utilizing advanced programming languages such as Python, SQL, and DBT.

• Optimize and enhance the performance of complex data workflows and queries to manage large data volumes while ensuring minimal latency.

• Support the design and implementation of data models for Data Warehouses, such as Snowflake.

• Establish data quality frameworks, including validation, monitoring, and testing, to guarantee data accuracy, consistency, and completeness.

• Set up comprehensive error logging and alerting mechanisms for monitoring pipeline health and addressing failures.

• Apply DevOps practices to data infrastructure, frequently utilizing Infrastructure as Code tools like Terraform.

⛳️ Requirements

• Over 5 years of experience in the Data industry.

• At least 4 years of experience in a Data Engineering position.

• Proficient in SQL for implementation within Data Warehousing systems and Data Pipelines.

• Experience with PostgreSQL, MySQL, Python, R, Redshift, and Snowflake.

• Background in database design and management.

• Strong understanding of data management principles, including data quality assurance and data governance.

• Familiarity with source code and version control systems.

• Experience with infrastructure automation tools, specifically Terraform.

• Knowledge of containerized services, such as Docker and Kubernetes.

• Experience in data transfer using various methods, including ETL/ELT, with tools like Airflow (MWAA), Argo, and dbt.

• Proficiency in using services within the AWS ecosystem, including RDS, EC2, Redshift, and EKS.

• Understanding of Linux commands, particularly in bash scripting.

• Ability to manage multiple projects, resolve challenges swiftly, deliver adaptable solutions, and appreciate diverse perspectives.

• Takes initiative, anticipates potential issues, tests solutions, and mentors others in effective problem-solving techniques.

• Sets quarterly objectives aligned with team goals, tracks progress in Workday, and communicates obstacles to the manager. Takes personal responsibility for the quality and timeliness of work, corrects mistakes, and learns from them.

• Demonstrates expert knowledge, introduces new tools and methodologies, and provides team training to enhance productivity.

• Tailors communications for audience comprehension, engages respectfully in team meetings, effectively synthesizes ideas, presents confidently, and actively listens to stakeholders.

• Consistently builds and maintains key stakeholder relationships, is regarded as a trusted partner, offers sound advice, and adeptly navigates challenging relationships.

🏝️ Benefits

• Two complimentary Getsmarter short courses per year.

• Subsidized medical aid through Discovery Health Medical Scheme.

• 4% contribution from 2U towards the Discovery Life Pension Fund and Group Risk Benefit.

• Access to the Employee Assistance Program (EAP).

• Generous leave policy that includes time off for volunteering with non-profit organizations, study leave, sports leave, and a company-wide festive season break.