Staff Data Engineer

📋 Description

• Design Petabyte Pipelines: Create scalable, reliable, and efficient data pipelines to compile extensive and complex datasets using SQL, DBT, and Snowflake, ensuring high availability and data integrity.

• Build Data Platforms: Take initiative to design and maintain internal React (TypeScript) interfaces and Python backend services that streamline data ingestion and discovery, cutting down lead times for application teams from weeks to mere minutes.

• Develop Data APIs: Construct and uphold production-quality REST and gRPC APIs that act as the high-performance bridge between our Snowflake data layer and downstream consumer interfaces.

• Modernize Data Operations: Adopt a GitOps framework for data utilizing GitHub Actions and Argo/Kargo, embedding standardized logging, alerting, and automated observability into every data product.

• Innovate with AI: Utilize Cursor AI, MCPs, and various AI tools to enhance the data engineering software development lifecycle, from optimizing complex SQL queries to automating schema migrations.

• Collaborate and Lead: Engage with business leaders to convert intricate data requirements into functional specifications while guiding fellow engineers in modern data architecture and software best practices.

• Data Architecture: Design and refine high-volume ETL/ELT pipelines employing SQL, DBT, and Snowflake, ensuring data is structured for both analytical and operational applications.

• Internal Tooling (Full Stack): Create and manage internal-facing web applications using React that empower data owners to engage with, monitor, and configure their data pipelines.

• API Development: Design and execute REST and gRPC APIs in Python that serve as the conduit between our Snowflake data layer and downstream consumer applications.

• CI/CD & GitOps: Oversee the deployment lifecycle of data services and tools utilizing GitHub Actions for continuous integration and Argo/Kargo for continuous delivery and lifecycle management.

• Self-Service Platforms: Develop "Data-as-a-Service" functionalities, such as automated UI-driven ingestion workflows, reducing dependency on manual data engineering requests.

• AI Integration: Leverage cutting-edge AI development tools (e.g., Claude AI) to expedite the creation of both data pipelines and management interfaces.

⛳️ Requirements

• Over 7 years of experience in Data Engineering with a strong emphasis on Python as the main scripting and backend language.

• More than 7 years of experience with SQL and cloud data warehouses (e.g., Snowflake, AWS, GCP, etc.).

• At least 7 years of experience in building high-volume ETL/ELT pipelines and data modeling.

• A minimum of 5 years of experience with DBT (Data Build Tools).

• Over 5 years of experience in developing frontend applications with React and designing RESTful APIs.

• At least 5 years of experience utilizing GitHub Actions and GitOps-based deployment tools (e.g., Argo or Kargo).

• Big Data Architecture: Broad understanding of big data design patterns, including Data Lake, Data Mesh, and Iceberg, along with data normalization techniques.

• GitOps & Deployment: Proven experience with Argo/Kargo for Kubernetes deployments and advanced GitHub Actions for workflow automation.

• Messaging & Streaming: Familiarity with message queuing technologies such as Kafka, SNS, or RabbitMQ to facilitate real-time data movement.

• AI-Enhanced Development: Skill in working with Cursor AI, GitHub CoPilot, or similar AI-driven environments to speed up engineering processes.

• Observability: Extensive experience with metrics, logging, monitoring, and alerting tools to ensure the reliability of production systems.

• Software Fundamentals: Strong understanding of data structures, algorithms, async programming patterns, and parallel programming.

• Healthcare Domain: Basic knowledge of HL7 V2.x or FHIR-based interface messages.

🏝️ Benefits

• Medical, dental, and vision coverage

• Paid time off

• Retirement savings options

• Wellness programs

• Other resources, based on eligibility