Senior Data Engineer – AI-Native Aftermarket Platform

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Create and develop resilient, idempotent data pipelines from the ground up using a modern data stack.

• Design both star and snowflake schemas, crafting precise, grain-aware SQL to build scalable data marts.

• Develop production-quality, unit-tested Python code at the module level, adhering to robust engineering practices such as type hinting and testing.

• Construct and validate dbt models across staging, intermediate, and mart layers while overseeing the overall project structure.

• Write and deploy jobs using Databricks Asset Bundles (DAB) in accordance with established architectural patterns.

• Implement thorough data quality checks at source, intermediate, and destination layers to avoid silent drops of nulls or duplicates.

• Ensure data governance through extensive dbt tests and a strict documentation-at-merge-time policy.

• Operate securely within a multi-repository architecture, utilizing service principals and guaranteeing no personal credentials are used in production deployments.

• Conduct cross-repository exposure checks before merging schema-altering changes.

• Take ownership of data pipelines from end to end, making critical technical design choices and mentoring mid-level engineers through meaningful code reviews.

• Establish the overall technical direction for core data systems, including modeling standards, branching strategies, observability thresholds, and secret management policies.

• Serve as a technical leader to remove obstacles for the team and actively engage in hiring panels to expand the engineering organization.

⛳️ Requirements

• Proficiency in SQL and dimensional modeling techniques, including medallion architecture, SCDs, and grain management.

• Demonstrated ability to design idempotent pipelines using incremental, checkpoint, and replaceWhere strategies.

• Extensive background in production-quality Python engineering, including type hints, pytest, and ruff.

• Strong aptitude for diagnosing and resolving failing Spark / PySpark jobs using tools like Spark UI.

• In-depth knowledge of Delta Lake features such as MERGE, OPTIMIZE, Z-ORDER, and time travel.

• Practical expertise with dbt, covering models, tests, and exposures.

• Experience in authoring and deploying jobs using Databricks Asset Bundles (DAB) and operating in a Unity Catalog environment.

• Commitment to data quality through pre-write asserts, schema validations, and the maintenance of dbt relationship and uniqueness tests.

• Strong adherence to disciplined Git workflows, conventional commit messages, and rigorous documentation standards.

• Experience with provisioning and utilizing Service Principals, GitHub environment secrets, and secret management solutions like Azure Key Vault or Databricks secret scopes.

• Excellent written technical communication skills for PR descriptions and runbooks, with the capability to translate pipeline efforts into business metrics.

• Proven decision-making skills to navigate ambiguity and balance trade-offs between cost, latency, and reliability.

• Experience in leading technical initiatives, establishing architectural standards, and contributing to interview rubrics is preferred.

• Familiarity with reading or modifying Azure Data Factory (ADF) pipelines and knowledge of Azure Data Lake storage is highly desirable.

• Knowledge of dbt observability tools, such as Elementary, is an advantage.

• Understanding of PII detection and masking best practices is preferred.

• Experience with multi-tenant configuration patterns to onboard new tenants without code changes is a strong plus.

• Proficiency in reading and editing GitHub Actions workflows for Databricks deployment is preferred.

• Ability to make cost-effective compute decisions, selecting the appropriate cluster shape for each workload, is a plus.

• Familiarity with AI-assisted development tools like Claude Code for daily tasks and code reviews is preferred.

• Experience in writing incident post-mortems and coordinating feature handovers with Data Science teams is a plus.

🏝️ Benefits

• 100% Remote Work: Enjoy the flexibility of working from a location that suits you best. All you need is a laptop and a reliable internet connection.

• Highly Competitive USD Pay: Receive an exceptional, market-leading salary in USD, exceeding typical market offerings.

• Paid Time Off: We prioritize your well-being. Our paid time off policies ensure you have the opportunity to relax and recharge when necessary.

• Work with Autonomy: Experience the freedom to manage your time effectively as long as the work is completed. Focus on results rather than hours clocked.

• Collaborate with Top American Companies: Enhance your skills by working on innovative, high-impact projects with industry-leading U.S. companies.