Data Engineer

This is a fully remote position, open to applicants in Spain.

📋 Description

• Design and develop robust, scalable data pipelines leveraging Databricks, Apache Spark, Delta Lake, and BigQuery.

• Create and implement efficient frameworks for data pipelines to ensure seamless data flow from various sources to data lakes, data warehouses, and analytical platforms.

• Diagnose and resolve issues associated with data processing, data quality, and the performance of data pipelines.

• Document data infrastructure, pipelines, and ETL processes to facilitate knowledge transfer and ensure smooth handovers.

• Develop automated tests and incorporate them into existing testing frameworks.

• Configure and optimize Databricks workspaces, clusters, and job scheduling effectively.

• Operate within a multi-cloud environment, including Azure, GCP, and AWS.

• Apply security best practices, including access controls, encryption, and audit logging.

• Build integrations with market data vendors, trading platforms, and risk management systems.

• Implement monitoring and performance tuning to maintain the health and efficiency of data pipelines.

• Collaborate with cross-functional teams to comprehend data requirements, pinpoint potential data sources, and define data ingestion processes.

• Work alongside data scientists, analysts, and other stakeholders to understand their data needs and deliver appropriate data solutions.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Data Engineering, or a related discipline (Master's degree preferred).

• A minimum of 3-5 years of experience in data engineering or full-stack development, specifically in cloud-based environments.

• Strong expertise in managing big data technologies such as Python, SQL, PySpark, and Spark.

• Demonstrated success in managing large-scale data projects.

• Extensive experience with Databricks.

• Proficient in database/backend testing with the capacity to write complex SQL queries for data validation and integrity assurance.

• Significant experience in designing and implementing data pipelines, ETL processes, and workflow automation.

• Familiarity with data warehousing concepts, dimensional modeling, data governance best practices, and cloud-based data warehousing platforms like Google BigQuery or Snowflake.

• Experience with cloud platforms such as Microsoft Azure or Google Cloud Platform (GCP).

• Experience working within a DevOps model.

• Proficient in various testing methodologies, including Unit, Functional, Integration, User Acceptance, System, and Security testing of data pipelines.

🏝️ Benefits

• JLL is committed to ensuring equal opportunities for both men and women.

• Opportunities for career advancement from within the company.

• Explore pathways to further develop your career internally.