Senior Data Engineer

This is a fully remote position, open to applicants in Illinois.

📋 Description

• Design, develop, and manage scalable data pipelines and capabilities for data platforms.

• Construct and maintain batch and near-real-time ETL/ELT pipelines.

• Integrate data from various sources including databases, APIs, applications, event streams, and external systems.

• Create reusable data models and curated data products for analytics, reporting, AI/ML, and operational applications.

• Develop intricate data transformations and processing workflows.

• Design data solutions that are scalable, available, resilient, efficient, and cost-effective.

• Develop reusable frameworks, libraries, and engineering patterns.

• Implement automated testing, CI/CD, Infrastructure as Code, and DataOps methodologies.

• Establish monitoring, alerting, data observability, and automated remediation functionalities.

• Diagnose production issues and conduct root-cause analysis for long-term enhancements.

• Optimize pipelines, queries, storage, and compute resources for performance and cost efficiency.

• Implement controls for data quality, validation, lineage, and reconciliation.

• Collaborate with Architecture, Governance, and Security teams to uphold enterprise standards.

• Ensure the secure and compliant handling of healthcare data.

• Provide technical leadership through solution design, architecture discussions, code reviews, and best practices.

• Mentor Data Engineers.

• Collaborate with Data Scientists and Machine Learning Engineers on datasets, feature pipelines, and AI/ML infrastructure.

• Work closely with Product, Engineering, Analytics, and business stakeholders.

• Maintain technical documentation for pipelines, models, architecture, and operational processes.

⛳️ Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related technical field.

• Over 5 years of experience in building production-grade data pipelines, data platforms, and large-scale data processing solutions.

• Advanced skills in Python and SQL.

• Solid foundation in software engineering principles.

• Practical experience with Snowflake and cloud-based data platforms.

• Familiarity with Kafka, Flink, or other distributed and streaming technologies.

• Extensive experience with AWS, including S3, Lambda, Glue, Kinesis, or equivalent cloud technologies.

• Experience in designing and optimizing ETL/ELT pipelines, data models, and high-volume data processing workflows.

• Experience with Airflow or similar orchestration platforms.

• Proficient in CI/CD, Git, automated testing, and Infrastructure as Code.

• Strong understanding of data quality, governance, security, observability, and production support.

• Excellent problem-solving, communication, and cross-functional collaboration skills.

• Full-stack mindset with a commitment to solving problems from end to end.

• Preferred: experience with dbt/dbt Cloud.

• Preferred: experience in AI-native engineering environments and AI-assisted development tools.

• Preferred: experience supporting Machine Learning and Generative AI workloads.

• Preferred: experience with Docker, Kubernetes, Terraform, or CloudFormation.

• Preferred: experience with data cataloging, lineage, metadata management, and data observability platforms.

• Preferred: experience in Healthcare, HealthTech, SaaS, or other regulated sectors.

🏝️ Benefits

• Health, dental/orthodontia, and vision coverage from day one.

• Synthetic ownership program after one year.

• Employee Assistance Program, including mental wellness and financial coaching.

• Generous paid time off (PTO).

• Team events, family picnics, and holiday festivities.

• Development programs aimed at career growth.

• Quarterly bonuses.

• 401(k) matching.