Principal Data Engineer

This is a fully remote position, open to applicants in California, +11 more states.

📋 Description

• Take ownership of the architecture and end-to-end delivery of modern data platforms.

• Develop client data strategies and create scalable solutions.

• Lead implementation efforts from the design phase through to production.

• Collaborate directly with client stakeholders, from gathering requirements to providing executive updates.

• Mentor internal team members to ensure the delivery of high-quality outcomes.

• Make pragmatic technology choices while consistently delivering high-quality work within set deadlines.

• Provide clients with guidance on architecture, scalability, and platform evolution, linking technical decisions to business results.

• Facilitate or initiate retrospectives and feedback discussions with teams and clients.

• Clearly articulate and document outcomes, driving action items to completion.

• Contribute positively to team dynamics and 8th Light's culture by embodying company values.

• Identify, mitigate, and resolve obstacles to team progress.

• Contribute through mentoring, leading workshops, representing 8th Light in the tech community, or aiding in hiring efforts.

⛳️ Requirements

• Over 10 years of experience in software development.

• More than 7 years of experience in data engineering.

• Proven long-term success in delivering data platforms.

• Progressed into technical leadership and architectural responsibilities.

• Complete ownership of modern data platform architecture and delivery.

• Extensive expertise in Databricks and modern lakehouse architecture, including Delta Lake and Spark-based processing.

• Advanced skills in SQL and Python for constructing and optimizing large-scale data pipelines.

• Hands-on experience with cloud platforms, data services, and infrastructure design.

• Strong background in data modeling, ETL/ELT patterns, and distributed data processing.

• Experience with orchestration tools and transformation frameworks in a production environment.

• Familiarity with client requirements gathering, solution design, and executive communication.

• Experience working with healthcare data formats such as Epic, HL7, FHIR, and claims.

• Knowledge of CI/CD, DevOps practices, and infrastructure-as-code tools.

• Experience in mentoring internal team members.

• Strongly preferred: experience in platform engineering, Kafka/Kinesis/Pub/Sub, production agentic systems including RAG and vector search, MLOps, and Infrastructure as Code.

🏝️ Benefits

• A comprehensive L&D program that offers unlimited access to thousands of books, courses, and expert-led training through the O'Reilly learning platform.

• An annual financial stipend and dedicated time for learning and development activities.

• Support for coworking arrangements for our global team.

• Flexible paid time off (PTO).

• 12 weeks of new parent leave available for eligible employees.

• Access to coworking spaces for team members located outside of hub cities.

• A remote-first work environment.