Senior GenAI Data Engineer, Databricks

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Design AI Agents: Create and implement agents capable of executing NLP-based data generation, automated data enrichment, and intricate data reasoning within Databricks.

• Natural Language Interfaces: Build "Chat with your Data" functionalities, empowering stakeholders to inquire about the data warehouse using natural language.

• Incorporate LLMs into data processes for enhanced automation and intelligence.

• Construct scalable data models to facilitate analytics and AI applications.

• Execute AI-driven improvements such as anomaly detection and data enrichment.

• Collaborate with data, analytics, and engineering teams to enhance data reliability.

• Enhance the performance and scalability of data and AI workflows.

• Support automation through CI/CD methodologies.

• Maintain data quality, traceability, and manageability across pipelines.

⛳️ Requirements

• Databricks & AI Architecture (Essential)

• - Robust experience with Databricks Lakehouse architecture.

• - Preferred expertise in Databricks Mosaic AI and Unity Catalog for managing AI assets.

• - Practical experience in developing RAG (Retrieval-Augmented Generation) pipelines utilizing Vector Search.

• SQL & Data Modeling (Essential)

• - Proficient in advanced SQL development.

• AI Engineering & Data Workflows (Essential)

• - Experience in integrating LLM APIs (OpenAI, Anthropic, etc.) into data workflows.

• - Practical experience using AI for:

• - Data enrichment.

• - Anomaly detection.

• - Automated classification.

• - Familiarity with LangChain, LlamaIndex, or comparable frameworks.

• - Awareness of Model Context Protocol (MCP) or similar methodologies for linking AI models with external tools and data sources.

• - Solid understanding of Tool Calling / Function Calling: enabling LLMs to securely interact with SQL databases and external APIs.

• - Experience in Prompt Engineering and Guardrailing: crafting system prompts that preserve context and hierarchy (e.g., understanding team associations).

• Platform & Engineering Practices (Preferred / Intermediate)

• - Familiarity with GitHub workflows.

• - Knowledge of CI/CD pipelines (Jenkins or equivalent).

• - Experience with YAML/YML configuration files.

🏝️ Benefits

• Monetary compensation.

• Year-end Bonus.

• IMSS, AFORE, INFONAVIT.

• Major Medical Expenses Insurance.

• Life Insurance.

• Funeral Expenses Coverage.

• TDU Membership.

• MediAccess.

• Health Check-Up Subsidy.

• Preferential rates for car insurance.

• Vacations.

• Official Mexican Holidays.

• Life Happens Days.

• Bereavement Leave.

• Civil Marriage Leave.

• English Classes.

• Certifications.

• Educational Agreements (Talisis, U-ERRE, UNID, TecMilenio, Tec de Monterrey, UDEM, SPIS).

• Corporate Agreements & Discounts (Sorteos Tec, Envia Flores, TopGolf).

• Taquitos Rewards.

• Birthday Bonus.

• Work-from-home Bonus.

• Laptop Policy.