Machine Learning Architect

This is a fully remote position, open to applicants in South Africa.

📋 Description

• Define and spearhead the architecture for scalable Machine Learning and AI platforms.

• Create comprehensive ML workflows utilizing Databricks, which encompass: Feature engineering, Model training, Experimentation, Deployment, and Monitoring.

• Design scalable data pipelines for AI/ML operations using technologies such as Apache Spark, Python, and SQL.

• Establish best practices for MLOps that include: CI/CD for ML, Model versioning, Model governance, Automated retraining, Model drifting, and Observability and monitoring.

• Develop secure and compliant AI architectures that adhere to governance and privacy standards.

• Collaborate with Data Engineering teams to enhance data models and feature stores.

• Mentor Data Scientists and ML Engineers on scalable production design patterns.

• Assess and incorporate contemporary AI capabilities, which include (this will be a plus): LLMs, Vector databases, Retrieval augmented generation (RAG), and AI agents.

• Promote cost efficiency, scalability, and operational excellence across ML platforms.

• Define reference architectures and best practices across numerous ML teams (not limited to a single project).

• Facilitate stakeholder engagement and convert business requirements into scalable technical solutions.

⛳️ Requirements

• Over 8 years of experience in Data, AI, or Machine Learning Engineering roles.

• More than 3 years of experience in designing ML platforms or AI architecture at scale.

• Strong practical experience with:

• - Databricks

• - Apache Spark

• - Python

• - SQL

• In-depth understanding of:

• - MLOps

• - ML lifecycle management

• - Distributed ML systems

• - Feature engineering

• - Model deployment methodologies

• Experience with Databricks Unity Catalog, Delta Lake, and Lakehouse architecture.

• Familiarity with cloud platforms (AWS, Azure, or GCP).

• Proven track record of deploying ML models in production settings.

• Strong expertise in data architecture and scalable ETL/ELT processes.

• Experience with orchestration frameworks like Apache Airflow.

• Excellent stakeholder communication and technical leadership abilities.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Flexible working hours and remote work opportunities.

• Professional development and continuous learning support.

• A collaborative and innovative work environment.