Associate Principal Engineer, Azure Data Factory Architect

📋 Description

• Design and construct scalable data integration solutions utilizing Azure Data Factory.

• Create, develop, and sustain ADF pipelines, triggers, datasets, and linked services.

• Execute ETL / ELT workflows for both batch and near real-time data processing.

• Develop and enhance ADF Mapping Data Flows.

• Integrate ADF with various Azure services such as Azure SQL, Synapse, Data Lake, and Blob Storage.

• Manage performance tuning, monitoring, and cost optimization of pipelines.

• Implement and oversee CI/CD pipelines for ADF utilizing Azure DevOps.

• Diagnose production issues and conduct root cause analysis.

• Establish architecture standards, best practices, and reusable frameworks.

• Mentor and support data engineers through hands-on assistance and code reviews.

• Guarantee data reliability, scalability, and high availability.

• Collaborate with stakeholders to gather data integration requirements.

⛳️ Requirements

• A minimum of 9 years of total experience.

• Extensive experience in Azure Data Engineering / Data Integration.

• Profound hands-on knowledge of Azure Data Factory (Essential).

• Strong grasp of ETL / ELT concepts (Essential).

• Practical experience with ADF pipelines, triggers, datasets, and linked services.

• Familiarity with Mapping Data Flows and Integration Runtimes (both Self-hosted & Azure).

• Significant experience with Azure Data Lake Storage (ADLS Gen2).

• Experience in integrating on-premise and cloud data sources.

• Strong SQL skills (Azure SQL / Synapse).

• Experience with Azure Synapse Analytics.

• Knowledge of CI/CD pipelines using Azure DevOps.

• Understanding of data orchestration, scheduling, and dependency management.

• Experience with monitoring, logging, and error handling in ADF.

• Good understanding of performance tuning and cost optimization.

• Strong problem-solving and communication abilities.

• Good to Have: Proficiency in Java (Strong).

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Opportunities for professional development and continuous learning.

• Flexible working hours and a supportive work environment.

• Comprehensive health and wellness benefits.

• Access to cutting-edge technology and tools.