Lead Data Engineer

This is a fully remote position, open to applicants in South Africa.

📋 Description

• Lead, recruit, and nurture a team of data engineers, focusing on performance management and career advancement.

• Own the complete delivery process, encompassing sprint planning, prioritization, and execution.

• Establish engineering standards to guarantee a high-quality and consistent output.

• Collaborate with cross-functional stakeholders to synchronize data initiatives with business objectives.

• Design and enhance scalable data architecture spanning ingestion, transformation, storage, and consumption layers.

• Develop and sustain high-volume ETL/ELT pipelines for both real-time and batch data processing.

• Lead practices related to data modeling, governance, and data quality.

• Promote the adoption of AI and automation in data engineering workflows.

• Collaborate with data science teams to operationalize models and integrate AI into data products.

• Assess and implement innovative tools, emphasizing practical impact.

⛳️ Requirements

• Bachelor’s degree with 8+ years of relevant experience (or Master’s degree with 5+ years).

• At least 2 years of experience in managing or leading engineering teams.

• Demonstrated experience in agile delivery and talent development within engineering.

• Strong skills in stakeholder management and effective communication.

• Extensive knowledge of modern data architecture, including data lakes, warehouses, and lakehouses.

• Experience with Snowflake and advanced SQL/data modeling techniques (e.g., Kimball, star schema).

• Proficient in Python, SQL, and Spark (with a preference for PySpark).

• Familiarity with streaming technologies such as Kafka.

• Practical experience with AWS services (S3, Glue, EMR, Lambda, Redshift).

• Knowledge of CI/CD, Infrastructure-as-Code (Terraform/CloudFormation), Airflow, Docker, and Kubernetes.

• Understanding of best practices in data governance, quality, and observability.

🏝️ Benefits

• Health insurance

• Retirement plans

• Paid time off

• Flexible work arrangements

• Professional development