Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Create, develop, and sustain scalable data engineering solutions utilizing Azure and Snowflake platforms

• Construct and enhance ETL/ELT pipelines with Azure Data Factory, Databricks, and Snowflake for both structured and unstructured data

• Build and maintain enterprise data warehouses, data marts, and analytical datasets for reporting, business intelligence, and advanced analytics

• Execute data transformation and orchestration using PySpark, SQL, Python, and Snowflake stored procedures

• Design and oversee cloud-based data lakes employing ADLS Gen2 and Delta Lake

• Work collaboratively with business stakeholders, data analysts, and architects to convert requirements into scalable data solutions

• Monitor and enhance pipeline performance for reliability, scalability, and cost-effectiveness

• Establish data quality frameworks, validation rules, and monitoring processes

• Administer Snowflake databases, schemas, warehouses, tasks, streams, and data-sharing functionalities

• Create dimensional and normalized data models

• Implement security measures, access management, encryption, and governance policies across Azure and Snowflake

• Assist in cloud migration and modernization from on-premises or legacy platforms

• Automate deployments utilizing Azure DevOps, Git repositories, and CI/CD pipelines

• Diagnose production issues, conduct root cause analysis, and resolve data-related obstacles

• Engage in Agile ceremonies, code reviews, and technical design discussions

• Generate technical documentation, architecture diagrams, operational runbooks, and knowledge-transfer resources

• Datavail offers data management, application development, analytics, and cloud services through a technology-enabled delivery platform

⛳️ Requirements

• Over 6 years of experience in designing, developing, and optimizing enterprise-scale data platforms utilizing Microsoft Azure and Snowflake

• Demonstrated expertise in constructing scalable data pipelines, data warehouses, and analytics solutions using Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Snowflake, Python, SQL, and Spark

• Proficient in data modeling, ETL/ELT development, cloud migration, performance tuning, data governance, and CI/CD methodologies

• Strong experience in integrating data from various sources to enable business intelligence and advanced analytics solutions

• Familiarity with Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Snowflake Data Cloud, Python, PySpark, SQL, ADLS Gen2, Azure Functions, Delta Lake, data warehousing, data modeling, ETL/ELT, CI/CD, Azure DevOps, Git, performance tuning, data governance, security, Power BI integration, and Agile/Scrum frameworks

• Microsoft Certified: Azure Data Engineer Associate (DP-203) is preferred

• SnowPro Core Certification is preferred

🏝️ Benefits

• Competitive salary and performance-based bonuses

• Comprehensive health, dental, and vision insurance

• Retirement savings plan with employer matching contributions

• Flexible work hours and remote work options

• Ongoing professional development and training opportunities

• Supportive and inclusive company culture