ETL Developer

This is a fully remote position, open to applicants in India.

📋 Description

• Create, design, and execute ETL workflows utilizing Azure Data Factory, Synapse Pipelines, Azure Data Lake, SQL, PySpark, and Synapse Spark Pools.

• Construct intricate ETL pipelines, data transformation logic, and integration processes for processing high volumes of data.

• Implement effective strategies for data extraction, transformation, and loading.

• Configure, set up, and deploy ETL systems to achieve optimal performance.

• Monitor workflows, resolve issues, and enhance data processing efficiency.

• Review and enhance pipelines to boost performance, scalability, and fault tolerance.

• Collaborate with business stakeholders, architects, and team members to pinpoint business logic and translate it into ETL processes.

• Design and execute business rules, data validation logic, and complex transformations.

• Ensure adherence to ETL design, coding standards, and best practices in data governance.

• Supervise code reviews and testing of ETL solutions.

• Document ETL designs, business logic, and workflow processes.

⛳️ Requirements

• 8–10 years of experience in data engineering and ETL development.

• Practical experience with Azure Data Factory (ADF), Synapse Pipelines, Azure Data Lake, T-SQL/SQL, PySpark, Synapse Spark Pools, Azure Data Lake Storage Gen2 (ADLS), Delta Lake, Azure Synapse Analytics, and Data Vault.

• Experience in designing, developing, and deploying ETL workflows using DevOps CI/CD pipelines and processes.

• Familiarity with high-performance data processing and integration strategies.

• Strong hands-on experience in setting up Synapse pipelines and managing extensive ETL processes.

• Solid understanding of data warehousing concepts.

• Advanced proficiency in SQL, including query optimization, complex transformations, and handling large datasets.

• Experience in implementing complex business logic, data validation, transformation, and enrichment within ETL processes.

• Knowledge of Azure SQL Database, Azure Data Lake, and Azure Blob Storage.

• Familiarity with ETL tools and technologies such as Azure Synapse Analytics, PySpark, Synapse Spark Pools, and SSIS (a plus).

• Experience with ADLS Gen2 and Delta Lake.

• Proven ability in tuning, troubleshooting, and optimizing high-volume ETL pipelines.

• Excellent skills in creating stored procedures, views, and functions.

• Strong communication and interpersonal skills.

• Exceptional problem-solving abilities.

• Capability to manage multiple priorities, adapt to evolving requirements, and meet deadlines.

🏝️ Benefits

• Remote work arrangement.

• Commitment to an equal opportunity and inclusive workplace.