Data Solution Architect

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Create production-ready data architectures utilizing the Databricks Lakehouse Platform.

• Develop and enhance Bronze, Silver, and Gold data pipelines.

• Diagnose and optimize large-scale Apache Spark jobs.

• Establish governance for data and AI assets through Unity Catalog.

• Implement access controls, security measures, and ensure data lineage.

• Collaborate with AWS, Azure, or GCP cloud infrastructures.

• Optimize the sizing of Databricks clusters, manage DBU usage, enhance performance, and reduce cloud costs.

• Potentially develop and oversee GenAI applications and machine learning models.

• Automate Databricks workflows and assist with streaming ingestion, data warehousing, and business intelligence integrations.

⛳️ Requirements

• Proficiency in designing comprehensive Lakehouse and Medallion architectures featuring Bronze, Silver, and Gold layers.

• In-depth knowledge of Databricks and Apache Spark internals along with distributed computing.

• Capability to troubleshoot and refine large-scale Spark jobs utilizing caching, partitioning, and broadcast joins.

• Familiarity with Delta Lake ACID transactions, schema enforcement, time travel, and Z-ordering.

• Experience in creating unified governance models for data and AI assets, including RBAC, row/column-level security, and data lineage.

• Strong expertise in a native cloud ecosystem: AWS, Azure, or GCP, encompassing VNet configurations and IAM roles.

• Advanced SQL capabilities.

• Proficiency in Python or Scala.

• Ability to monitor DBUs, appropriately size serverless and multi-node clusters, and optimize cloud costs and performance.

• Advanced English proficiency required.

• Must reside in Mexico.

• Databricks certifications are desirable but not mandatory.

• Experience with GenAI, MLflow, CI/CD, DevOps, streaming data, data warehousing, and BI tools is a plus.

🏝️ Benefits

• Cookies and consent-management options are available on the careers site; no specific benefits related to the role are mentioned.