Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Collaborate with both technical and non-technical stakeholders to convert business needs into scalable, reliable, and maintainable data engineering solutions.

• Create, develop, and sustain data integration solutions utilizing Microsoft SQL Server, Azure Data Factory, Azure Databricks, Microsoft Fabric, and associated Azure services.

• Construct, manage, and enhance ETL/ELT pipelines with Azure Data Factory and Microsoft Fabric Pipelines, including aspects like pipeline orchestration, parameterization, scheduling, monitoring, and error handling.

• Create and uphold data transformation logic through Azure Databricks notebooks and Microsoft Fabric notebooks, emphasizing performance, scalability, reusability, and operational reliability.

• Design and execute data ingestion, transformation, validation, and delivery processes across structured, semi-structured, and external data sources.

• Develop and refine data integration procedures leveraging Azure Data Factory, Azure Databricks, Azure Storage Accounts, Azure Data Lake Storage, Azure Key Vault, Microsoft Fabric Lakehouses, Warehouses, Pipelines, and Notebooks.

• Build and maintain SQL-based data models, stored procedures, views, automation scripts, and database objects to support operational and analytical workloads.

• Assist in the modernization and migration of legacy data integration processes from SSIS and on-premises platforms to cloud-based Azure and Microsoft Fabric architectures.

• Design and implement reusable pipeline patterns, data quality checks, validation controls, logging, alerting, and performance monitoring processes.

• Facilitate new client onboarding efforts by creating and executing reliable data ingestion, transformation, reconciliation, and validation workflows.

• Diagnose and resolve data pipeline, data quality, performance, and integration challenges in collaboration with teams within Intelligence & Analytics.

• Design, develop, and maintain datasets, semantic models, and data structures that underpin reporting, analytics, and visualization solutions across Power BI, Excel, Microsoft Fabric, and other business intelligence platforms.

• Contribute to data engineering standards, documentation, best practices, and reusable frameworks that enhance scalability, maintainability, and operational efficiency.

• Investigate, assess, and suggest emerging technologies and platform capabilities that bolster data architecture, automation, performance, and reliability.

⛳️ Requirements

• Over 3 years of experience in data engineering, data integration, business intelligence engineering, or a similar technical role.

• More than 3 years of experience in writing, optimizing, and troubleshooting SQL, including T-SQL, stored procedures, views, indexing strategies, and query performance tuning.

• Practical experience in designing, building, and supporting ETL/ELT pipelines with Azure Data Factory and/or Microsoft Fabric Pipelines.

• Direct experience in developing data processing and transformation logic using Azure Databricks notebooks, Microsoft Fabric notebooks, PySpark, Spark SQL, Python, or comparable technologies.

• Solid understanding of Microsoft SQL Server and relational database design principles.

• Familiarity with Azure data platform services like Azure Data Factory, Azure Databricks, Azure Storage Accounts, Azure Data Lake Storage, Azure Key Vault, and related cloud data services.

• In-depth knowledge and experience with Microsoft Fabric components such as Pipelines, Notebooks, Lakehouses, Warehouses, Dataflows, and semantic models.

• Experience in constructing reliable data ingestion and transformation processes from APIs, flat files, databases, web services, cloud storage, and other external data sources.

• Hands-on experience with implementing data validation, reconciliation, monitoring, logging, and alerting processes for production data pipelines.

• Experience in supporting production data workloads, including troubleshooting failures, optimizing performance, and enhancing pipeline reliability.

• Previous experience in modernizing or migrating legacy ETL processes from SSIS, SQL Server, or on-premises environments to Azure or Microsoft Fabric is highly preferred.

• Bachelor’s degree in Computer Science, Information Technology, Data Analytics, a related field, or equivalent professional experience.

• Familiarity with Power BI, semantic models, or other data visualization and analytics tools is advantageous.

• Experience in Customer Data Platforms, CDPs, database marketing, marketing analytics, or client data onboarding environments is a plus.

• Basic knowledge of AWS or Google Cloud Platform is an added benefit.

• Strong analytical, problem-solving, and critical thinking abilities.

• Excellent verbal and written communication skills, capable of explaining technical concepts to both technical and non-technical audiences.

• Ability to independently handle moderately complex data engineering tasks while effectively collaborating in a team-oriented environment.

🏝️ Benefits

• Comprehensive medical coverage with a choice of 3 plans, along with dental and vision insurance.

• Company-funded short-term disability, long-term disability, and life insurance.

• Optional supplemental life, accident, and critical illness insurance plans.

• 401K plan with company matching contributions.

• Flexible PTO and a generous holiday schedule.

• Fully paid primary caregiver leave (12 weeks) & parental bonding leave (2 weeks).