Senior Data Engineer – DevOps, DataOps, Delivery Engineering

This is a fully remote position, open to applicants in Illinois.

📋 Description

• Assist in the daily operation and oversight of data pipelines supporting over 10 concurrent client environments.

• Establish, configure, and facilitate the deployment of client-facing applications, from initial environment setup through go-live stabilization and continuous support.

• Validate the integrity of daily data flows, data loads, and refreshes across client environments.

• Aid in the provisioning and configuration of new client environments, including SFTP ingestion, instance setup, access control, file formats, scheduling, and encryption.

• Contribute to the development of Jenkins CI/CD pipelines, manage AWS infrastructure, and oversee Posit Connect administration.

• Create and maintain monitoring, alerting systems, data integrity alerts, and data volume quality control dashboards.

• Investigate issues related to processing, applications, and data quality; conduct root cause analysis and implement remediation strategies.

• Develop and maintain client-specific ingestion pipelines and standardized data validation scripts.

• Ingest historical data for onboarding new clients and address data quality incidents.

• Assist in the migration of the data lake and core tables to Snowflake.

• Manage AWS Lambda functions and event-driven triggers.

• Provision client instances and oversee database setup, ingestion configuration, and access controls.

• Manage SAML/SSO certificate administration and PGP encryption setup.

• Implement repository governance, deployment controls, and code review processes.

• Establish and validate development and staging environments.

• Support Docker-based monorepo deployment architecture.

• Execute and document processes for client offboarding.

• Facilitate compliance activities, including PHI classification, audit documentation, certificate renewals, access restrictions, and credential rotation.

• Maintain operational runbooks and documentation.

• Report to and work under the technical guidance of the DataOps lead.

⛳️ Requirements

• Authorization to work in the U.S. is mandatory.

• A Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent experience) is required.

• 2-3+ years of experience in data engineering, DevOps, or DataOps roles, specifically with hands-on experience in supporting production data pipelines.

• Proficient in AWS data pipeline infrastructure — including S3, Athena, Lambda, EC2, and event-driven architectures (SNS/SQS).

• Strong Python scripting skills for data ingestion, validation, and pipeline automation, along with familiarity in Bash.

• Experience with CI/CD pipelines (Jenkins or similar) and Git-based version control practices (branch protection, PR workflows).

• Background in setting up, configuring, and supporting application deployments.

• Experience managing multi-client or multi-tenant data environments, which includes onboarding and provisioning new environments.

• Participation in incident response and root cause analysis for data pipeline and application issues is preferred.

• Comfortable working as a generalist across infrastructure, data engineering, and DevOps.

• Familiarity with Snowflake or other modern cloud data warehouse platforms.

• Understanding of Posit Connect or R/Shiny environments.

• Experience with Docker and containerized deployment.

• Knowledge of SAML/SSO certificate management and cloud security practices.

• Experience in healthcare data environments, including SFTP clinical file delivery, PHI handling, and HL7/ADT.

• Proven experience in building monitoring and alerting systems for data pipelines.

• Some experience in coordinating directly with client IT teams.

🏝️ Benefits

• Annual incentive compensation program.

• Medical coverage.

• Dental coverage.

• Vision coverage.

• Wellness programs.

• Huron benefit plans.