Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Oversee the design, development, and enhancement of enterprise data ingestion pipelines, transformation logic, and data platform elements within Microsoft Fabric.

• Create and implement scalable ETL and ELT architectures for both structured and semi-structured enterprise data.

• Configure, manage, and sustain Microsoft Fabric tenants, workspaces, capacities, security, and governance.

• Execute the Bronze, Silver, and Gold medallion architecture layers.

• Develop transformation processes that include schema validation, cleansing, normalization, enrichment, and reconciliation.

• Set up data quality monitoring, validation controls, automated testing, and alerting systems.

• Design environment separation across development, testing, staging, and production stages.

• Construct and maintain CI/CD deployment pipelines for data platform artifacts.

• Implement infrastructure-as-code for consistent, version-controlled deployments.

• Develop and enhance Spark, PySpark, SQL, and associated data-processing workloads.

• Configure and maintain encryption, RBAC, Microsoft Entra ID integration, Privileged Identity Management, and least-privilege controls.

• Facilitate secure integration with enterprise source systems and hybrid on-premises/cloud connectivity.

• Diagnose and optimize issues related to pipelines, platforms, performance, connectivity, and data quality.

• Assist in disaster recovery and business continuity planning; develop and test rollback, recovery, and restoration procedures.

• Create reusable engineering patterns, standards, and technical documentation.

• Collaborate with architects, governance specialists, cybersecurity teams, application teams, and source-system owners.

• Engage in architecture reviews and propose modernization and integration solutions.

• Mentor Data Engineers and Associate Data Engineers; evaluate implementations and offer technical guidance.

• Provide senior-level problem-solving support and facilitate knowledge transfer and operational transition.

⛳️ Requirements

• Extensive experience in designing and implementing enterprise data pipelines, ETL/ELT processes, and cloud-based data platforms.

• Practical experience with Microsoft Fabric.

• Experience in administering or configuring enterprise cloud data environments.

• Strong background with Azure Data Factory or similar data integration technologies.

• Proficiency in SQL.

• Experience with Spark and PySpark for large-scale data processing and transformation tasks.

• Familiarity with implementing enterprise ETL/ELT frameworks.

• Experience in designing and executing medallion architectures using Bronze, Silver, and Gold data layers.

• Knowledge of schema validation, data cleansing, normalization, reconciliation, and data quality controls.

• Experience in developing CI/CD pipelines for data platform deployments.

• Experience in implementing infrastructure-as-code.

• Familiarity with source control, automated deployment, and environment promotion processes.

• Understanding of cloud security principles, including RBAC, identity integration, encryption, and least-privilege access.

• Capability to troubleshoot complex data pipeline, platform, and integration issues.

• Experience in supporting production data environments with availability, reliability, and performance requirements.

• Strong technical documentation and communication skills.

• Experience in mentoring or providing technical leadership to other engineers.

• Active Secret clearance.

🏝️ Benefits

• Company-supported certifications aligned with current and future program work, including cloud, Kubernetes, DevSecOps, security, AI/ML, project management, and related technical fields.

• Clear advancement pathways and future leadership opportunities.

• 401(k) plan with a 100% match up to 6%.

• Comprehensive medical, dental, vision, life, and disability insurance coverage.

• Generous PTO and paid holidays.

• Home-office equipment plan and support for remote work.

• Fitness and wellness reimbursement.

• Weekly pay schedule.

• Modern perks, including team events.