Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Optimize, support, and implement cloud-driven data systems that are essential for mission-critical healthcare applications.

• Develop secure, scalable, and compliant database solutions utilizing Azure SQL Server and various cloud-native technologies.

• Aid in the advancement of the analytics lakehouse and its associated data sources.

• Design and construct resilient data pipelines and lakehouse solutions with the help of Databricks, Microsoft Fabric, PySpark, and Python.

• Create AI-ready schemas, semantic layers, and metadata structures for precise AI-agent data inquiries.

• Establish automated validation processes for queries generated by agents.

• Initiate agentic automation for workflows related to data quality and reconciliation, measuring accuracy and ensuring rollback strategies are in place.

• Curate and manage the semantic layer to minimize incorrect responses from AI agents.

• Catalog and oversee enterprise data assets for analytics, financial planning, and product engineering purposes.

• Collaborate with Product Engineering, FP&A, and other stakeholders to identify and comprehend data requirements.

• Diagnose performance issues and provide support for SQL Server and Azure SQL environments.

• Mentor junior engineers in data engineering best practices, architectural patterns, and technical standards.

• Contribute to reviews of data protection, security, and compliance, including relevant HIPAA requirements.

• Ensure that activities adhere to EEO laws, HIPAA, ERISA, and other applicable regulations.

• Undertake additional responsibilities as assigned based on business needs.

⛳️ Requirements

• Over 7 years of production experience in data engineering roles, including practical experience in building and maintaining large-scale data pipelines.

• Advanced skills in Python and PySpark or Apache Spark within production data engineering environments.

• Practical, hands-on experience with Databricks, Microsoft Fabric, or similar modern data platforms.

• Proven experience in designing, implementing, and managing lakehouse architectures using Delta Lake, Parquet, or comparable technologies.

• Experience in designing AI-ready schemas, semantic layers, or metadata structures with consideration for downstream AI or machine learning query patterns, or experience in creating automated validations for programmatically generated queries.

• Familiarity with agentic or automation-driven data quality workflows, such as AI-assisted reconciliation or agent-grounded semantic layers, or similar applied solutions.

• Demonstrable ability to work directly with product, analytics, finance, or business stakeholders to proactively understand and shape data requirements.

• Knowledge of Microsoft Azure cloud concepts, including platform as a service, infrastructure as a service, Azure SQL, and hybrid cloud models.

• Willingness and capability to develop deeper operational expertise within Azure SQL and SQL Server environments.

• Significant experience mentoring junior or mid-level engineers on technical standards, architectural patterns, and data engineering best practices.

• Preferred: Hands-on production experience with Azure SQL, SQL Server, or Azure SQL Managed Instance.

• Preferred: Proficiency in T-SQL and PowerShell for automation and operational support.

• Preferred: Experience with infrastructure-as-code tools such as Terraform, Bicep, or Azure Resource Manager templates.

• Preferred: Knowledge of streaming data architectures and technologies like Azure Event Hubs or Apache Kafka.

• Preferred: Understanding of data mesh principles and distributed data ownership.

• Preferred: Experience in healthcare or another regulated industry.

• Preferred: Familiarity with HIPAA requirements or healthcare data standards such as FHIR and HL7.

• Must be eligible to work in the United States.

• Willingness to travel as needed, not exceeding 10%.

🏝️ Benefits

• Competitive annual bonus opportunities.

• 401(k) plan with employer matching.

• Flexible Time Off: take leave when necessary without concern for available hours.

• 11 paid holidays each year.

• Your Day Your Way - Celebrate a day of cultural or social significance that matters to you.

• Access to LinkedIn Learning courses for ongoing professional development.

• Variety of Medical, Dental, and Vision plan options.

• Health Savings Account with employer matching contributions.

• Flexible Spending Account available.

• 100% company-paid parental leave after 6 months of employment.

• Company-paid life insurance and short/long-term disability insurance.

• Nextech Luminary Peer Recognition Program.

• Employee Assistance Program offering free counseling sessions.

• Corporate discounts on retail, travel, and entertainment.