Senior Data Engineer, Spark

This is a fully remote position, open to applicants in Egypt.

📋 Description

• Develop and operate large-scale batch data pipelines.

• Ensure data processing tasks remain swift and cost-effective as data volumes increase.

• Create and oversee extensive Apache Spark data processing tasks.

• Conduct performance optimization to ensure efficient and dependable pipeline execution.

• Assist with data modeling for analytical purposes.

• Structure data in a lakehouse for future use.

• Manage automated scheduling, monitoring, and data quality verification.

• Collaborate with platform and product teams on comprehensive data workflows.

⛳️ Requirements

• Proficient hands-on experience with Apache Spark, including performance optimization, not merely through a managed notebook.

• Experience in data modeling for analytics and structuring data in a lakehouse for downstream usability.

• Knowledge of automated scheduling, monitoring, and data quality verification.

• Experience collaborating with platform and product teams on comprehensive data workflows.

• Familiarity with Apache Iceberg or other open table formats.

• Experience with Trino or similar query engines.

• Background in on-premises or self-managed cluster environments.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Opportunities for professional development and career advancement.

• Flexible working hours and remote work options.

• Supportive and inclusive company culture.