Market Data Engineer – Domain Trading Expertise Required

This is a fully remote position, open to applicants in Spain.

📋 Description

• Take ownership of the entire capture process from wire to data lake.

• Develop both batch and streaming pipelines (using Airflow, Spark, dbt) for tick and reference data.

• Manage L2/L3 order-book reconstruction, including gap handling.

• Oversee the Iceberg-over-S3 lakehouse for data storage and modeling.

• Enhance storage cost efficiency through techniques such as compaction, tiering, and snapshot expiration.

• Create libraries for schema management, data contracts, validation, and data lineage.

• Integrate monitoring, alerting, SLAs/SLOs, and CI/CD into the capture and pipeline layers on Kubernetes to ensure reliability and observability.

• Collaborate with Quant Research, Data Science, Backend, and DevOps teams to convert requirements into platform functionalities.

⛳️ Requirements

• Over 5 years of experience in developing production-grade data systems.

• Demonstrated ability to design and implement data lakes/lakehouses from the ground up.

• Practical knowledge of Apache Iceberg or similar table formats such as Delta or Hudi.

• Familiarity with market data and/or network packet capture.

• Experience in normalizing market data from various vendors.

• Proficient in Python (including libraries like Polars and/or PySpark); knowledge of Rust is a significant advantage.

• Competence in modern orchestration (Airflow) and distributed processing (Apache Spark).

• Advanced SQL skills, including complex aggregations, window functions, query optimization, and partition pruning.

• Strong foundation in Linux, containerization (Docker, Kubernetes/EKS), and cloud object storage (AWS S3).

• Knowledge of DevOps and observability practices including CI/CD, infrastructure-as-code (Terraform), and GitOps (ArgoCD).

• Solid understanding of structured, unstructured, and binary data along with storage optimization techniques.

• Fluency in English for effective documentation and collaboration within an international team.

🏝️ Benefits

• Health insurance compensation.

• Compensation for sports activities.

• Funding for professional development opportunities.

• Fully remote work with the flexibility to operate from anywhere in the world.

• Flexible scheduling to accommodate work-life balance.