Senior Data Architect

This is a fully remote position, open to applicants in Latin America.

📋 Description

• Design and oversee the complete lakehouse architecture on Databricks utilizing Medallion (bronze/silver/gold) patterns.

• Develop and enhance scalable batch and streaming pipelines using PySpark, Spark SQL, and Python.

• Lead platform migrations, including transitions from legacy data warehouses or Hadoop/EMR to Databricks, achieving measurable reductions in cost and runtime.

• Establish data governance frameworks, incorporating Unity Catalog, fine-grained access control, auditing, lineage, and compliance measures.

• Model data for analytical and operational purposes across OLTP/OLAP systems, dimensional, SQL, and NoSQL models.

• Set engineering standards for CI/CD, unit and integration testing, code reviews, reusable Python packages, and Databricks Asset Bundles.

• Manage infrastructure as code using Terraform and implement FinOps practices to optimize cloud expenditures.

• Assess and prototype AI use cases involving LLMs, vector databases, agents, and MLflow.

• Provide architectural guidance to other teams and mentor junior and mid-level engineers.

• Communicate design trade-offs effectively to both engineers and non-technical stakeholders.

⛳️ Requirements

• Residing in Latin America and available for remote work.

• Fluent in English; proficiency at C1 level or above.

• Over 10 years of experience in data engineering, with at least 5 years of hands-on experience with Databricks and Spark in production environments.

• Strong expertise in Python and PySpark; familiarity with Scala and advanced SQL.

• Proven experience in designing data lakes, data warehouses, and lakehouses, having led at least one significant migration.

• Practical experience on at least one major cloud platform: AWS, Azure, or GCP.

• Hands-on experience with data governance, including Unity Catalog, access controls, lineage, and auditing.

• Solid software engineering principles: Git workflows, automated testing, CI/CD (Azure DevOps, GitHub Actions, or similar tools).

• Proficiency in infrastructure as code using Terraform.

• Ability to clearly articulate design trade-offs to both technical and non-technical stakeholders.

• Experience mentoring engineers.

• Proficiency in Spanish or Portuguese is advantageous.

• Databricks certifications are a plus.

• AWS Solutions Architect or equivalent cloud architecture certification is a plus.

• Multi-cloud experience across AWS, Azure, and GCP is a plus.

• Experience creating agentic or RAG solutions with LangChain, LangGraph, MLflow, OpenAI APIs, and vector databases is a plus.

• Experience with real-time and IoT data processing at high volumes is a plus.

• Background in finance, telecommunications, payments, or SaaS data domains is a plus.

🏝️ Benefits

• Full-time contract via a payroll platform.

• Remote work assigned at the client location.

• Salary ranging from USD $7,000 to $8,800 per month, based on seniority and skillset.