Lead Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design and develop scalable and secure Azure data architectures tailored for business intelligence, analytics, and operational systems.

• Create and sustain data pipelines utilizing Azure Data Factory, Azure Synapse, and related services.

• Contribute to the design of databases across Azure SQL Database, SQL Managed Instance, PostgreSQL, and CosmosDB.

• Optimize storage strategies for Azure Blob Storage, Data Lake, and structured databases.

• Craft data models for effective analytics and reporting.

• Develop stored procedures, functions, triggers, and views.

• Implement database DevOps processes, encompassing schema versioning, deployment automation, and rollback procedures.

• Conduct database performance tuning, indexing, and query optimization.

• Ensure data integrity, security, and compliance with HIPAA/HITRUST regulations for PHI data.

• Manage the lifecycle of databases across development, pre-production, and production environments using RedGate Flyway.

• Design and execute ETL/ELT processes that integrate various data sources.

• Create event-driven processing solutions leveraging Azure Service Bus and Event Grid.

• Integrate external REST APIs and develop internal Python data APIs using FastAPI/Flask on Azure Container Apps or Functions.

• Develop data transformation logic and orchestration workflows.

• Ensure data quality, consistency, and lineage throughout the data platform.

• Integrate Azure AI services and machine learning endpoints as required.

• Review Power BI semantic models, reports, and dashboards.

• Translate business requirements into technical solutions in collaboration with stakeholders.

• Provide technical guidance and mentorship to junior developers and engineers.

• Engage in architecture reviews and contribute to technical decision-making.

• Collaborate with DevOps and Infrastructure teams to enhance the platform.

• Promote continuous improvement in data quality, performance, and reliability.

• Maintain technical documentation, data dictionaries, architecture diagrams, process flows, lineage, transformation logic, and business rules.

• Keep audit trails and compliance documentation up to date.

• Troubleshoot complex data issues and offer production escalation support.

• Participate in an on-call rotation as required.

• Build EHR integrations utilizing FHIR, C-CDA, and HL7 when necessary.

• Define OpenAPI specifications, API versioning, and conduct contract tests.

• Lead document templating and HTML-to-PDF rendering pipelines.

• Report directly to the Director of Data Engineering.

⛳️ Requirements

• Bachelor’s degree in computer science, Information Systems, Data Engineering, or a related field, or equivalent professional experience.

• Over 8 years of experience in data engineering, database development, or similar roles.

• Expert-level skills in T-SQL and Microsoft SQL Server.

• Proficient in Python at an expert level.

• Experience in building REST APIs and background workers using Python (FastAPI/Flask).

• Understanding of event-driven architectures and asynchronous processing patterns.

• Extensive experience with Azure data services (Azure SQL, Data Factory, Synapse, Blob Storage, etc.).

• Practical experience in constructing and maintaining data pipelines and ETL/ELT processes.

• Familiarity with Power BI development, including DAX, data modeling, and performance optimization.

• Solid grasp of database design principles, normalization, and data modeling.

• Experience with Git and CI/CD methodologies.

• Microsoft certification in Azure Data Engineer Associate, Azure Database Administrator, or Power BI Data Analyst.

• Knowledge of Infrastructure as Code (Terraform, Bicep, ARM templates).

• Familiarity with Azure Container Apps, Functions, and Logic Apps.

• Experience with PostgreSQL and CosmosDB.

• Understanding of HIPAA/HITRUST compliance requirements.

• Proficiency in PowerShell for automation and scripting tasks.

• Familiarity with Azure DevOps or similar CI/CD platforms.

• Experience with data governance, cataloging, and metadata management.

• Background in healthcare technology or regulated industries.

• Experience in designing OpenAPI/Swagger specifications and conducting contract testing.

• Proficient with advanced AI development tools such as Claude Code and GitHub Copilot.

• Knowledge of Azure API Management and private networking practices.

• Willingness to travel up to 15% as necessary.

• Hands-on experience with FHIR and C-CDA parsing/transformation.

• Experience integrating with Epic, Cerner, Athena, or other EHR/EMR systems.

• Experience with HL7 v2 message handling.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Retirement savings plan with company matching.

• Opportunities for professional development and continuing education.

• Flexible work schedule and remote work options.

• Generous paid time off and holidays.

• Collaborative and innovative work environment.