Senior Data Engineer

This is a fully remote position, open to applicants in Egypt.

📋 Description

• Evaluate various structured and unstructured data sources to gain insights into schemas, data quality, and interrelationships.

• Analyze and document metadata, formats, and business rules associated with incoming datasets.

• Conduct thorough data profiling to determine completeness, accuracy, uniqueness, and consistency.

• Create logic to match and reconcile records across diverse systems using keys, rules, or fuzzy matching methodologies.

• Design and execute processes for the creation and upkeep of golden master records.

• Assist in the establishment and maintenance of master data across customer, product, asset, and other domains.

• Manage data deduplication and standardization efforts.

• Construct scalable data pipelines utilizing Azure Data Factory for data ingestion, transformation, and cleansing.

• Ensure efficient data transit across staging, processing, and analytical layers.

• Utilize Azure Synapse Analytics for sophisticated data processing and analytics.

• Collaborate with business analysts, data stewards, and application owners to comprehend data definitions and ownership.

• Uphold data governance standards and best practices.

• Supervise data workflows and enhance performance for large-scale processing.

• Diagnose data issues and implement effective logging and monitoring solutions.

⛳️ Requirements

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related discipline.

• Over 5 years of experience in data engineering, ETL development, or data integration.

• Demonstrated expertise in data profiling, matching, and Master Data Management (MDM).

• Proficient in SQL for querying, transforming, and optimizing complex datasets.

• Strong skills in Python or PySpark for data processing and pipeline development.

• Experience with Azure Data Factory for constructing and managing data pipelines.

• Familiarity with Azure Synapse Analytics is an advantage.

• Extensive knowledge of ETL/ELT tools (e.g., Talend, Informatica, SSIS, Apache NiFi).

• Understanding of data quality frameworks and metadata management.

• Excellent communication skills in both Arabic and English.

• Experience with data cataloging tools (e.g., Azure Purview, Alation).

• Knowledge of data modeling and normalization practices.

• Awareness of data privacy and compliance regulations (GDPR, HIPAA, etc.).

• Familiar with REST APIs and integrating with third-party data services.

🏝️ Benefits

• Comprehensive health and wellness programs.

• Opportunities for professional development and career advancement.

• Flexible work arrangements and a supportive work environment.