Data Engineer

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Design, develop, and support solutions for data replication and integration utilizing HVR.

• Create, build, and sustain scalable data pipelines using Databricks (Spark, Delta Lake).

• Develop and enhance ETL/ELT processes for both structured and unstructured data.

• Work with extensive datasets to ensure data quality, integrity, and performance optimization.

• Implement data models and transformations for analytics and reporting purposes.

• Collaborate with data scientists and analysts to facilitate advanced analytics and machine learning workloads.

• Integrate data from various sources, including databases, APIs, and streaming systems.

• Optimize Spark jobs for enhanced performance and cost efficiency.

• Enforce data governance, security measures, and access controls.

• Monitor and troubleshoot data pipelines along with any production issues.

• Support CI/CD pipelines and adhere to DevOps best practices for data engineering tasks.

• Assist in data migration and modernization projects.

• Produce operational documentation, runbooks, and support procedures.

• Engage in production support, issue resolution, and performance tuning efforts.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Engineering, or a related discipline.

• Practical experience in data engineering or big data development.

• Strong proficiency in SQL, Oracle, and SQL Server.

• Familiarity with Java and JavaScript (preferred).

• Knowledge of web-based applications and APIs (REST/SOAP).

• Significant experience with the Databricks Platform.

• Proficiency in Apache Spark (PySpark/Scala).

• Competence in SQL and Python.

• Experience with Delta Lake and data lake architectures.

• Practical knowledge of cloud platforms (Azure, AWS, or GCP).

• Understanding of data orchestration tools (Azure Data Factory, Airflow, etc.).

• Familiarity with data warehousing concepts (Star schema, Snowflake schema).

• Experience with version control systems (Git) and CI/CD pipelines.

• Strong grasp of data pipeline optimization and performance tuning.

🏝️ Benefits

• Flexible work arrangements.

• Professional development opportunities.