Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of the health, reliability, and architectural strategy of the data pipeline utilized by care teams, provider partners, and analytics functions.

• Design and manage pipelines and warehouse structures, primarily focusing on the flow of payer data into Diverge Health.

• Ensure the maintenance of the data pipeline, emphasizing the health and timeliness of payer data.

• Ingest and standardize new member rosters and claims data from payers, creating models to interpret payer structures and integrate data into a unified view.

• Make architectural decisions concerning database design, schema, and Snowflake environment structure, while balancing performance, cost, and scalability.

• Manage Snowflake infrastructure, including provisioning, account and user creation, as well as access and role management.

• Establish and support Snowflake as a service for various teams across the organization, including application development.

• Oversee the Git environment, including repository and permissions management, setup, and governance.

• Develop and sustain data pipelines from file receipt to processing, ensuring reliability, graceful failure handling, monitoring, and alerting.

• Participate in the team's rotating support schedule and address issues during business hours.

• Assist in iterative definitional work with stakeholders by building and testing proposed definitions against data and presenting the impacts.

• Independently lead a team or project, providing technical guidance and mentorship.

• Collaborate with Finance, Payer Account Management, and other teams to comprehend data needs, align definitions, and convey technical concepts and trade-offs.

• Assess process improvements and implement solutions to enhance development and optimize pipeline performance.

⛳️ Requirements

• A minimum of 10 years of professional data engineering experience, with a proven history of independently managing production data pipelines.

• Experience in the healthcare industry, including practical work with claims data, membership data, and healthcare data transformation.

• Proven hands-on expertise with Dagster as the primary orchestration tool.

• Expert-level proficiency in Python.

• Extensive hands-on experience with Snowflake, encompassing schema design, performance optimization, and environmental structure decisions.

• Experience in Snowflake administration, including provisioning, account and user creation, access and role management, as well as supporting Snowflake as a service for other teams.

• Strong SQL skills and a broad understanding of data warehousing fundamentals beyond any single platform.

• Experience in Git administration, including repository and permissions management, setup, and governance.

• Demonstrated ability to lead and mentor fellow engineers, serving as a technical resource for the team.

• Comfort in operating in situations where definitions are not yet established, with the judgment to create them in collaboration with the business rather than waiting for existing answers.

• Strong communication skills, with the ability to clearly explain technical concepts to non-technical audiences.

• Proven capability to balance competing priorities and ensure work is completed in a dynamic, fast-paced environment.

• Preferred: Bachelor's degree or equivalent experience in computer science, engineering, or a related technical field.

• Preferred: Experience with Salesforce development.

• Preferred: Proficiency in VS Code or similar development tools.

• Preferred: Experience with AI coding assistants such as Claude, ChatGPT, or Copilot integrated into a professional development workflow.

• Candidates must be authorized to work in the United States without the need for employer sponsorship, now or at any time in the future.

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Comprehensive health and wellness benefits.

• Opportunities for professional development and continuous learning.

• Flexible work schedule and remote work options.

• Supportive and collaborative team environment.