Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and maintain data pipelines utilizing Microsoft Fabric, Azure Data Factory, and associated Azure technologies.

• Create Python, PySpark, and SQL notebooks for data ingestion, transformation, and processing across Bronze, Silver, and Gold layers.

• Integrate data from various sources including files, databases, APIs, and enterprise applications.

• Execute data transformations, enrichment, aggregations, and apply business rules.

• Enhance pipelines and SQL queries for improved performance, reliability, and scalability.

• Develop and uphold enterprise Data Products in alignment with Data Governance.

• Manage on-premises SQL Server databases, stored procedures, and ETL processes while facilitating migration to Azure and Microsoft Fabric.

• Create and sustain automated, rule-based processing workflows, encompassing queue-driven processing and batch orchestration.

• Oversee file-based ingestion, feed tracking, processing status, and downstream data distribution.

• Implement data validation, reconciliation, logging, monitoring, and exception handling.

• Monitor production pipelines, address failures and data discrepancies, and enact corrective measures.

• Ensure the continuity, reliability, and prompt delivery of business-critical data processes.

• Collaborate with business stakeholders, Data Governance, application teams, and engineering resources.

• Transform business requirements into technical designs while identifying source data gaps, dependencies, and areas for process improvement.

• Maintain technical documentation that includes data flows, source-to-target mappings, transformation logic, and operational procedures.

• Conduct unit and integration testing, support QA and business sign-off, and adhere to development, deployment, and Data Governance standards.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Software Engineering, Information Technology, or a related technical field.

• 3–5+ years of practical experience in data engineering, database development, or enterprise data integration, including pipeline construction, data migration, and environment modernization.

• Strong expertise in SQL Server, T-SQL, stored procedures, query optimization, and ETL/ELT development.

• Practical experience with Azure Data Factory, Azure Data Lake Storage, and Microsoft Fabric, including Data Pipelines, Lakehouse, and Notebooks.

• Proficiency in Python and/or PySpark.

• Understanding of data warehousing, relational data modeling, and Medallion Architecture.

• Experience with file-based integrations, batch processing, queue-driven workflows, data validation, monitoring, troubleshooting, and production support.

• Familiarity with SSIS or legacy ETL tools is a plus.

• Experience in modernizing legacy data platforms is a plus.

• Relevant Microsoft certifications are advantageous.

• Excellent communication and collaboration abilities.

• Strong analytical and problem-solving skills, with a focus on data accuracy, quality, and operational reliability.

• Cover letters are highly encouraged.

🏝️ Benefits

• Comprehensive and highly competitive benefits package.

• Physical health benefits.

• Retirement and savings benefits.

• Caregiving benefits.

• Emotional wellbeing benefits.

• Transportation benefits.

• Elective benefits tailored to meet employees’ needs and personal situations.

• Reasonable accommodation for qualified individuals with disabilities during the application or interview process.