Senior Data Consultant, Microsoft Fabric

This is a fully remote position, open to applicants in Alabama, +18 more states.

📋 Description

• Assist in modernizing a client's existing Azure Synapse data platform and migrating prioritized workloads to Microsoft Fabric.

• Collaborate with Trility consultants and client engineers to enhance Fabric architecture.

• Create reusable, metadata-driven ingestion frameworks.

• Transition data workloads into a contemporary lakehouse environment.

• Develop and orchestrate data pipelines and Spark notebooks in Fabric using PySpark and Spark SQL.

• Analyze Azure Synapse pipelines and convert processing logic into Fabric solutions.

• Establish data validation and reconciliation processes, including duplicate detection, key alignment, and record-level comparisons.

• Suggest full-load, incremental, or change data capture patterns.

• Implement Git-based CI/CD and automate environment promotion through GitHub Actions, Azure DevOps, or comparable tools.

• Utilize Claude Code to define implementation specifications, quicken development, and assess AI-generated code.

• Review data dictionary, metadata, lineage, and catalog needs, including potential Microsoft Purview solutions.

• Work closely with client engineers through paired development, code reviews, and knowledge-sharing sessions.

• Apply industry best practices for performance, testing, monitoring, documentation, security, and operational support.

⛳️ Requirements

• Must reside within the United States.

• Must be authorized to work in the United States without sponsorship, both now and in the future.

• Must successfully pass a background check.

• 5+ years of experience with Azure Data Services, including Microsoft Fabric, Azure Data Factory, Databricks, or Synapse.

• Practical experience with Microsoft Fabric in production settings, covering lakehouses, pipelines, notebooks, medallion architecture, workspaces, and capacity management.

• Strong background in building and maintaining production ETL/ELT pipelines and reusable data ingestion frameworks.

• Proven experience in designing and implementing a metadata-driven ingestion framework, including configuration-based onboarding and extensible processing patterns.

• Experience in developing Spark notebooks and data transformation workflows using PySpark and Spark SQL.

• Proficient SQL skills for data transformation, validation, and performance tuning, with the ability to comprehend existing T-SQL logic.

• Familiarity with Azure Synapse, including the analysis of existing pipelines and understanding complex data processing dependencies.

• Experience implementing medallion architecture patterns (Bronze, Silver, Gold).

• Proven track record in designing and implementing Git-based CI/CD workflows for data platform artifacts using GitHub Actions, Azure DevOps, or similar tools.

• Hands-on experience with Claude Code for AI-assisted software development, including directing code generation, working across multiple files, and critically evaluating generated code.

• Experience in validating data migrations and ensuring data quality, accuracy, and consistency between source and target systems.

• Experience in integrating data from REST APIs, relational databases, and structured or semi-structured data sources.

• Familiarity with data governance, lineage, metadata management, and data catalog concepts.

• Ability to collaborate effectively with technical teams, constructively challenge existing approaches, and engage in client-facing technical discussions.

• Strong written and verbal communication skills.

• W2 employees must reside in Alabama, Colorado, Delaware, Florida, Georgia, Iowa, Kansas, Kentucky, Louisiana, Minnesota, Missouri, Montana, Nebraska, Nevada, North Carolina, Ohio, South Carolina, Tennessee, Texas, Virginia, or Wisconsin.

🏝️ Benefits

• Work remotely from anywhere in the United States with flexible working hours.

• Paid time off, company holidays, and floating holidays.

• Comprehensive benefits with contributions toward premiums.

• 401(k) plan.

• Sponsored and supported learning opportunities.