Data Solution Architect

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Create comprehensive, production-ready data architectures and pipelines utilizing Lakehouse and Medallion Architecture.

• Diagnose and optimize large-scale Spark jobs.

• Develop integrated data and AI governance frameworks.

• Set up and manage cloud infrastructure across AWS, Azure, or GCP.

• Build data solutions leveraging advanced SQL and Python or Scala.

• Oversee Databricks Units, adjust cluster sizes, and enhance performance and cloud expenditures.

• Potentially incorporate GenAI and MLflow, along with CI/CD and DevOps practices, streaming data, data warehousing, and BI tools.

⛳️ Requirements

• Proficient in creating end-to-end data architectures using Lakehouse and Medallion Architecture, encompassing Bronze, Silver, and Gold layers.

• In-depth knowledge of Databricks and Spark internals as well as distributed computing.

• Capability to troubleshoot and optimize large-scale Spark jobs through techniques like caching, partitioning, and broadcast joins.

• Familiarity with Delta Lake ACID transactions, schema enforcement, time travel, and Z-ordering.

• Demonstrated expertise in formulating unified governance models for data and AI resources, including RBAC, row-level security, column-level security, and data lineage.

• Strong understanding of AWS, Azure, or GCP cloud environments, including VNet configurations and IAM roles.

• Advanced SQL proficiency and fluency in Python or Scala.

• Ability to monitor DBUs, optimize serverless and multi-node clusters, and manage cloud expenses effectively.

• Advanced conversational skills in both English and Spanish; both are mandatory and will be assessed.

• Candidates based in Mexico must currently reside in the country.

• Foreign nationals living in Mexico must possess a valid INE, CURP, NSS, and RFC.

• Must have an RFC with a homoclave and a current Constancia de Situación Fiscal issued by Mexico's SAT.

🏝️ Benefits

• Exceptional superior benefits.