Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership and enhance the data pipeline architecture encompassing ingestion, transformation, modeling, and serving.

• Make independent architectural decisions at the project level while assessing trade-offs related to freshness, cost, scalability, and simplicity.

• Spearhead platform-level enhancements for managing warehouse costs, optimizing compute efficiency, and refining access control architecture.

• Identify and spearhead technical initiatives that involve orchestration, CI/CD, data access, and improving developer experience.

• Lead large-scale technically complex projects or manage multiple concurrent medium-sized initiatives across various teams.

• Direct the monitoring and testing strategy for the domain.

• Address observability gaps, establish alerting before failures occur, and resolve the most challenging production issues.

• Influence technical decisions and the architectural direction beyond the Data & Analytics team.

• Collaborate with EPD stakeholders regarding infrastructure decisions that impact their project roadmaps.

• Mentor data engineers and analytics engineers.

• Evaluate architectural and modeling decisions.

• Incorporate AI into data engineering workflows through the use of tools and automation.

• Guide others in the effective and validated use of AI.

⛳️ Requirements

• Over 7 years of experience in data engineering.

• Proven history of managing end-to-end pipeline architecture.

• Extensive experience in designing orchestration workflows utilizing Apache Airflow.

• Proficient in containerized data infrastructure deployed in production environments.

• Experienced in deploying services, diagnosing operational challenges, and enhancing platform reliability and scalability.

• Skillful in evaluating and conveying architectural trade-offs to both technical and non-technical stakeholders.

• Background in building or significantly refining CI/CD practices for data pipelines.

• Experience with automated testing, validation, and deployment processes.

• Demonstrated ability in leading incident response and monitoring strategies.

• Expertise in establishing alerting and observability measures prior to failures.

• Proven ability to influence technical decisions across various teams or functions.

• Experience in collaborating with engineering, product, or data science stakeholders outside of the immediate team.

• Experience in mentoring fellow data engineers.

• Familiarity with Kubernetes-based data infrastructure.

• Proven experience in leading comprehensive migrations from legacy ETL to modern orchestration.

• Knowledge of observability and monitoring tools such as Datadog.

• Experience in developing internal tools or automation, including AI-assisted tools.

🏝️ Benefits

• Equity.

• Benefits and perks are available, although specific details are not listed.