Lead Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Make immediate contributions to projects and quickly establish trust with clients.

• Design and construct data pipelines and schemas that remain effective as data volume and complexity increase.

• Lead discussions on data architecture and modeling, evaluate trade-offs, and document the rationale.

• Pragmatically balance technology choices while delivering high-quality results within set deadlines.

• Generate solutions for both technical and non-technical challenges in collaboration with teammates and clients.

• Facilitate or initiate retrospectives and feedback discussions with teams and clients.

• Clearly articulate and document outcomes, driving action items forward.

• Contribute to the team and organizational culture by exemplifying 8th Light's Values.

• Maintain relationships with client technical leaders.

• Identify, mitigate, and resolve obstacles hindering team progress.

• Contribute through mentoring, leading workshops, representing 8th Light in the tech community, or aiding in recruitment.

⛳️ Requirements

• Typically, 6 or more years of experience in data engineering.

• Design and build scalable data pipelines that ingest, transform, and move data from EHRs, claims, APIs, and CRMs.

• Hands-on experience with at least one cloud data platform: Snowflake, Databricks, Azure Data Factory, AWS Redshift, or Google BigQuery.

• Proficient in production SQL, including query optimization and debugging.

• Experience with Python or another programming language such as Scala or Java for automating data workflows.

• Knowledge of ETL/ELT principles, data warehousing, and data modeling.

• Experience delivering projects on AWS, Azure, or GCP.

• Ability to quickly learn unfamiliar tools.

• Proven experience in shipping high-quality, maintainable software collaboratively and incrementally.

• Familiarity with Git and CI/CD practices.

• Capability to communicate data decisions and trade-offs to both technical and non-technical stakeholders.

• Experience in mentoring and knowledge sharing.

• Strongly preferred: healthcare data experience with Epic, HL7, FHIR, or claims.

• Strongly preferred: experience with CI/CD pipelines, Git, and DevOps workflows.

• Strongly preferred: Infrastructure as Code expertise using Terraform or CloudFormation.

• Strongly preferred: knowledge of real-time and streaming data tools such as Kafka, Kinesis, or Pub/Sub.

• Strongly preferred: containerization experience with Docker and Kubernetes.

• Strongly preferred: cloud or data platform certifications.

🏝️ Benefits

• A comprehensive L&D program offering unlimited access to thousands of books, courses, and expert-led training on the O'Reilly learning platform.

• Annual financial stipend and designated time for learning & development.

• Coworking support for our global team.

• Flexible PTO.

• 12 weeks of parental leave available for eligible employees.