Data Engineer – MS Fabric

This is a fully remote position, open to applicants in Mexico.

📋 Description

• You will play a crucial role in designing and implementing modern data architectures of high quality, driving analytical solutions based on Big Data technologies.

• You will design, maintain, and optimize parallel processing systems, applying best practices for storage and management in data warehouses, data lakes, and lakehouses.

• You will collect, process, clean, and orchestrate large volumes of data, understanding both structured and semi-structured models.

• You will define the optimal strategy based on business objectives and technical requirements.

• You will execute development activities by consistently applying best data practices and the technologies we implement.

• You will identify requirements and define the scope, participating in sprint planning and engineering sessions.

• You will proactively collaborate in workshops and meetings with the internal team and the client.

• You will meet committed deadlines and manage risks by communicating deviations in a timely manner.

⛳️ Requirements

• Advanced English proficiency.

• Technical Skills: Query and Programming Languages T-SQL / Spark SQL: DDL and DML, intermediate and advanced queries (subqueries, CTEs, multiple joins with business rules), grouping and aggregation (GROUP BY, window functions, business metrics), stored procedures for ETL/ELT, index optimization, statistics, and execution plans for massive processes.

• Python (PySpark): Object-oriented programming (classes, modules), management of structures and data types (variables, lists, tuples, dictionaries), flow control using conditionals and loops, ingestion of structured and semi-structured data, development of DataFrames and UDFs, temporal windows and partitioning for optimization, good coding practices (PEP8, modularity).

• JSON / REST APIs: Orchestration of pipelines and CI/CD deployments through calls to Fabric REST APIs, dynamic execution parameterization, and artifact management.

• Microsoft Fabric Lakehouse (OneLake + Delta Lake): Data modeling with Delta ACID tables, partitioning and optimizations (OPTIMIZE, Z-ORDER) to enhance performance; use of time travel for auditing and recovery.

• Warehouses (Synapse Analytics): Configuration of provisioned SQL clusters and serverless; design of star/snowflake schemas; execution of transactional T-SQL with isolation and automatic resource scaling.

• CI/CD & Lifecycle Management: Definition of pipelines in Azure DevOps or GitHub Actions with dev-test-prod environments; unit testing of datasets, schema validations, and automated deployment of artifacts.

• Monitor Hub & Activator: Creation of custom dashboards for ingestion and transformation metrics (latency, throughput, errors); proactive alerts and automated runbooks based on defined conditions.

• Eventstreams & Eventhouse: Configuration of no-code real-time event ingestion; definition of processing windows, incremental aggregations, and optimized storage for temporal analysis.

• Data Security and Governance: Granular management of roles (Admin, Member, Contributor, Viewer) and permissions by workspace/item; row-level, column-level security, and dynamic data masking policies; access and change auditing for regulatory compliance.

• Desirable: General knowledge of Azure Data Factory.

🏝️ Benefits

• WELLNESS: We will promote your overall well-being through personal, professional, and financial balance.

• LET'S RELEASE YOUR POWER: You will have the opportunity to specialize comprehensively in various areas and technologies, achieving interdisciplinary development.

• WE CREATE NEW THINGS: We like to think outside the box. You will have the space, trust, and freedom to create, along with the training necessary to achieve it.

• WE GROW TOGETHER: You will participate in cutting-edge technological projects, multinational teams, and collaborate with foreign teams.