Databricks Solution Architect

This is a fully remote position, open to applicants in India.

📋 Description

• Create and execute scalable data pipelines utilizing Delta Live Tables (DLT), Spark SQL, Python, or Scala.

• Enhance ETL, streaming, and machine learning workloads to improve performance, cost-effectiveness, and reliability.

• Manage and set up Databricks Workspaces, Unity Catalog, and cluster policies to ensure secure and governed environments.

• Automate infrastructure and deployment processes using Terraform, Git, and CI/CD pipelines.

• Establish observability, cost optimization, and monitoring frameworks with tools such as Splunk, Prometheus, or CloudWatch.

• Work collaboratively with clients to develop AI and LLM solutions utilizing MLflow, DBRX, and Mosaic AI.

⛳️ Requirements

• Extensive hands-on experience with Databricks, covering workspace configuration, notebooks, clusters, and job orchestration.

• In-depth knowledge of Delta Lake, DLT, Unity Catalog, and SQL Warehouses.

• Skilled in Python or Scala for data engineering and machine learning workflows.

• Strong comprehension of AWS, Azure, or GCP cloud environments.

• Experience with Terraform automation, as well as DevOps and MLOps methodologies.

• Knowledge of monitoring and governance frameworks applicable to large-scale data platforms.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance plans.

• Flexible working hours and remote work options.

• Opportunities for professional development and continuous learning.

• Collaborative and inclusive company culture.