Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Influence the evolution of the data interoperability platform.

• Design and manage secure, high-performance data pipelines for both real-time and batch public health data transfers.

• Create and implement scalable solutions that ensure the reliable movement of sensitive health data while adhering to strict security and compliance regulations.

• Develop and enhance ETL/ELT pipelines utilizing NiFi, Kafka, Python, and SQL.

• Establish and sustain CI/CD pipelines for NiFi flows, Python, and SQL using GitHub and GitHub Actions, including flow versioning through NiFi Registry and automated promotion through development, testing, and production stages.

• Deploy and manage data pipelines on Azure and AWS in collaboration with DevOps/SRE teams.

• Implement CI/CD methodologies, including pull request reviews, branch protection, automated testing and validation, parameterized environment configuration, and versioned releases with rollback capabilities.

• Work alongside interoperability, product, and data science teams to transition legacy pipelines to modern architectures.

• Optimize data lakes and guarantee seamless data exchange across systems.

• Establish monitoring and alerting mechanisms, including Grafana dashboards and alerts, and produce operational runbooks.

• Resolve production issues to ensure stable and high-performing pipelines.

• Foster continuous improvement throughout the data ecosystem.

• Create and deliver high-quality, scalable integrations.

• Share expertise and facilitate smooth, efficient development workflows.

⛳️ Requirements

• Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related discipline.

• 3+ years of experience implementing enterprise-level data pipelines, or 7+ years of relevant professional experience that encompasses building and operating production data pipelines.

• Proficient hands-on experience with NiFi, Kafka, Python, and SQL, including the design, operation, and optimization of ETL/ELT pipelines in cloud environments.

• Demonstrated capability to construct robust, production-grade pipelines from scratch and work effectively with distributed data systems.

• Practical experience deploying and managing data pipelines within Azure or AWS, ensuring secure handling of sensitive data (IAM/RBAC, encryption, secrets management).

• Experience in applying CI/CD best practices for data pipelines using GitHub and GitHub Actions, inclusive of NiFi flow versioning and promotion (NiFi Registry), automated testing, and peer-reviewed releases.

• Ability to collaborate with DevOps/SRE to integrate pipelines into existing tools.

• Familiarity with or experience integrating OpenSearch/Elasticsearch.

• Practical experience with relational databases and established schemas.

• Strong grasp of data security and compliance requirements related to sensitive data.

• Excellent collaborator across functions with clear communication and strong technical documentation abilities.

• A curious, proactive problem solver with a growth-oriented mindset who excels in fast-paced, agile environments and is driven by mission-focused work and complex interoperability challenges.

• Preferred or standout experience includes healthcare data standards such as HL7 v2.x and FHIR; LOINC, CVX, SNOMED, and ICD-10; event-driven architectures; data de-identification; Rhapsody or Mirth; Docker or Kubernetes; Terraform, Bicep, or CloudFormation; Grafana, Azure Monitor, or CloudWatch; scaling production data platforms; and Master Patient Index or data deduplication frameworks.

🏝️ Benefits

• Virtual first, remote organization and culture.

• Flexible Paid Time Off (PTO).

• 401(k) retirement plan with corporate matching.

• Medical, prescription, vision, and dental coverage (multiple plans based on your needs).

• Short Term and Long Term Disability (for employee).

• Life Insurance (for employee).

• New Team Member support for home office setup.