Principal Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of the architecture and end-to-end delivery for modern data platforms.

• Influence the data strategy of clients and create scalable solutions.

• Lead the implementation process from design to production.

• Collaborate directly with client stakeholders from the requirements phase to executive updates.

• Mentor internal team members to ensure high-quality delivery.

• Make pragmatic technology choices while delivering high-quality work within set deadlines.

• Provide clients with advice on architecture, scalability, and platform evolution, linking technical decisions to business outcomes.

• Facilitate or initiate retrospectives and feedback discussions with teams and clients.

• Clearly articulate and document outcomes while driving action items forward.

• Enhance team and company culture by exemplifying 8th Light’s Values.

• Identify, mitigate, and resolve obstacles to team progress.

• Contribute through mentoring, leading workshops, representing the tech community, or assisting with hiring efforts.

⛳️ Requirements

• Over 10 years of experience in software development and more than 7 years in data engineering.

• Proven track record of successfully delivering data platforms over the long term.

• Clear advancement into technical leadership and architectural responsibilities.

• Extensive expertise in Databricks and contemporary lakehouse architecture, including Delta Lake and Spark-based processing.

• Proficient in SQL and Python for managing large-scale data pipelines.

• Practical experience with cloud platforms, data services, and infrastructure design.

• Strong background in data modeling, ETL/ELT patterns, and distributed data processing.

• Experience with orchestration tools and transformation frameworks in a production setting.

• Ability to lead technical delivery while remaining hands-on.

• Client-facing experience in gathering requirements, designing solutions, and communicating with executives.

• Skilled in navigating ambiguity and effectively prioritizing tasks.

• Familiarity with healthcare data such as Epic, HL7, FHIR, and claims.

• Knowledge of CI/CD, DevOps practices, and infrastructure-as-code tools.

• Experience in mentoring internal team members.

• Actively seeks and receives feedback.

• Strongly preferred: experience in platform engineering; real-time and streaming tools such as Kafka, Kinesis, or Pub/Sub; production agentic systems including RAG pipelines and vector search; MLOps; Infrastructure as Code.

🏝️ Benefits

• Unlimited access to a vast library of books, courses, and expert-led training on the O'Reilly learning platform.

• Annual financial stipend and allocated time for learning & development.

• Coworking support available for our global team.

• Flexible paid time off (PTO).

• 12 weeks of new parent leave offered for eligible employees.