Big Data Developer

This is a fully remote position, open to applicants in Spain.

📋 Description

• Design, develop, and maintain data lake pipelines utilizing Apache Spark, Scala, and SQL.

• Create and uphold scalable data processing solutions for handling large data volumes.

• Manage batch workflows and dependencies through Control-M and Airflow orchestration.

• Execute data ingestion, transformation, validation, and publication processes tailored for Market Risk applications.

• Assess and enhance Spark workload performance, reliability, and scalability.

• Evaluate technical and functional requirements, identify discrepancies, and propose effective resolutions.

• Participate in delivery planning and collaborate with various teams.

• Provide support for production incidents and conduct root-cause analyses.

• Aid in the ongoing improvement of operational processes.

• Work in tandem with Agile squads, functional teams, and production support units.

• Employ software engineering and data development best practices to deliver robust, maintainable, and production-ready solutions.

⛳️ Requirements

• Minimum of 2 years of professional experience in Big Data, Data Lakes, or cloud-based data platforms.

• Strong practical experience with Apache Spark, Scala, and SQL for large-scale data processing.

• Solid understanding of Data Lake architecture, data quality controls, data lineage, and production-grade data pipelines.

• Excellent analytical and problem-solving abilities.

• Outstanding communication and teamwork skills.

• Capability to collaborate with both technical and functional stakeholders.

• Adaptability and a proactive stance towards addressing technical challenges.

• English proficiency at a C1 level or equivalent professional fluency.

• Experience with workflow orchestration and scheduling tools, especially Control-M and Airflow, is highly valued.

• Familiarity with Databricks, Azure, and AWS is highly valued.

• Knowledge of YARN and cloud-based compute environments is highly valued.

• Experience with CI/CD and software engineering practices, including GitHub, Jenkins, and SonarQube, is highly valued.

• Background in banking production environments, operational monitoring, and incident management tools such as ServiceNow is highly valued.

• Understanding of Market Risk processes, including P&L, valuations, risk metrics, and regulatory reporting data flows, is highly valued.

🏝️ Benefits

• Full-time, permanent employment contract.

• 100% remote work with the flexibility to operate from any location.

• Opportunities for professional growth and career development.

• Access to ongoing learning resources.

• A multicultural and international work environment.

• Private medical insurance.

• Life insurance coverage.

• Lunch and transport cards provided through a flexible remuneration program.

• Online language courses available.

• Smart Office Pack to facilitate remote working.