Data Engineering Lead

This is a fully remote position, open to applicants in Pennsylvania.

📋 Description

• Lead, mentor, and cultivate a team of two to four data engineers.

• Establish clear priorities, plan projects, conduct code reviews, guide team members, and facilitate career growth.

• Take ownership of platform strategy across Azure and Microsoft Fabric.

• Identify redundant pipelines, storage, and tools, and develop a roadmap for consolidation or retirement of unnecessary components.

• Construct and enhance the Microsoft Fabric platform, encompassing Lakehouse, Warehouse, Data Factory pipelines, and OneLake.

• Determine which elements should remain in Azure and which should transition to Fabric.

• Create complex T-SQL queries, stored procedures, and database solutions.

• Oversee, monitor, and optimize SQL Server database environments.

• Design, optimize, test, and maintain stored procedures that support essential ETL processes.

• Manage transformation pipelines from end to end, including orchestration, scheduling, monitoring, failure handling, troubleshooting, and recovery.

• Lead impact analysis and change management for significant upstream source-system modifications.

• Collaborate with platform owners and downstream stakeholders.

• Establish and uphold standards for data quality, testing, observability, documentation, security, and CI/CD.

• Partner with analysts, BI developers, technical teams, and business stakeholders to convert requirements into dependable, scalable data products.

• Manage platform reliability, performance, cloud costs, access, lineage, compliance, and PII handling.

• Assess emerging tools and methodologies and provide recommendations on build versus buy decisions.

• Communicate priorities, risks, trade-offs, and progress to both technical and non-technical leaders.

• Report directly to the Director of Data Operations.

⛳️ Requirements

• Five or more years of experience in data engineering.

• At least two years of experience in a leadership role for engineers, either formally or as a technical lead.

• Strong, hands-on experience with Microsoft Fabric in a production setting.

• Familiarity with Azure Data Factory, Synapse, ADLS Gen2, and Azure SQL.

• Advanced T-SQL expertise, including writing, tuning, testing, and maintaining stored procedure-based ETL at scale.

• Experience in orchestrating and monitoring transformation pipelines.

• Ability to identify and resolve production failures.

• Proficient in Python.

• Understanding of Spark or PySpark.

• Knowledge of dimensional and lakehouse or medallion data modeling.

• Awareness of contemporary ETL and ELT design patterns.

• Experience with CI/CD and infrastructure as code.

• Proficiency in tools such as Azure DevOps or GitHub Actions.

• Experience with Terraform or Bicep.

• Excellent communication skills, sound judgment, and ability to influence stakeholders while balancing hands-on delivery with team leadership.

• Preferred certifications include Microsoft Certified Fabric Analytics Engineer Associate DP-600 or Fabric Data Engineer Associate DP-700.

• Experience managing data from Salesforce, ATS or VMS platforms, as well as ERP or finance systems is preferred.

• Preferred experience in leading a data-platform migration or consolidation.

• Familiarity with applying AI within the data estate is preferred.

• Successful completion of a background check is required.

🏝️ Benefits

• Health, dental, and vision coverage.

• Flexible Spending Account (FSA).

• 401(k) plan.

• Health Reimbursement Account (HRA).

• Health Savings Account (HSA).

• Life insurance coverage.

• Disability insurance.

• Paid parental leave.

• Holiday pay.

• Vacation and sick leave.

• Recognized company holidays.

• Up to 3 floating holidays.

• Opportunities for community and social events.

• Learning and development initiatives.

• Contribution of $100 per month towards student loan payments.

• Tuition reimbursement opportunities.

• Access to Employee Assistance Program.

• Other health and wellness benefits.

• Annual performance-based bonus opportunity of 10% of base salary.

• Flexible spending accounts.

• Health savings accounts.

• Employee tuition reimbursement.

• Student debt assistance.

• Reasonable accommodation for individuals with physical and mental disabilities.