Senior Data Engineer

This is a fully remote position, open to applicants in Connecticut, +2 more states.

📋 Description

• Take ownership of pipelines, data models, and ensure end-to-end platform reliability across Snowflake's ingress and egress paths.

• Address platform incidents, conduct root cause analysis, and implement lasting solutions.

• Establish and report on pipeline service levels for data freshness, job success rates, and recovery times.

• Offer debugging, triage, support, and design advice to engineers and analysts.

• Engage in on-call and event support rotations, with increased coverage during significant live events.

• Design and develop shared platform components, including medallion layers, role and access taxonomy, ingestion libraries, and orchestration frameworks.

• Lead the consolidation of Snowflake, which encompasses replication, account merges, secure-share repointing, parity validation, and cutover.

• Phase out redundant legacy tools such as Airflow, notebook-driven ingestion, SSIS, and outdated CI/CD processes.

• Create golden paths, templates, reference implementations, and documentation to facilitate onboarding of sources and deployment of models.

• Maintain shared Python ingestion connectors and observability hooks.

• Manage Fivetran ingestion and make decisions regarding build versus buy options.

• Oversee Prefect on EKS, including deployments, work pools, worker health, containerized flows, and flow taxonomy.

• Administer Snowflake as a shared multi-tenant platform, managing performance, replication, secure sharing, and warehouse costs.

• Declaratively manage Snowflake objects using Terraform.

• Construct and maintain AWS infrastructure with Terraform, covering EKS, networking, IAM, S3, Lambda, and secrets management.

• Maintain GitHub Actions CI/CD with AWS OIDC, promotion paths, review gates, dbt tests, and data quality checks.

• Develop and review dbt models, consolidating macro libraries.

• Collaborate with Analytics Engineering and Data Science on governed semantic models, feature pipelines, and reporting datasets.

• Implement masking, tokenization, row-access policies, network policies, secrets rotation, and SSO-driven provisioning as code.

• Assist with cataloging, lineage, access reviews, audits, and security remediation.

• Prepare the platform for ML, AI, feature pipelines, and datasets ready for retrieval.

• Utilize AI-assisted engineering tools and contribute to establishing team best practices.

• Mentor global delivery partners and engineers, review work against established standards, and provide guidance to stakeholders and leadership on technical trade-offs.

⛳️ Requirements

• 7+ years of experience in building and operating production data platforms.

• Minimum of 3 years concentrating on shared platform and infrastructure work.

• Extensive hands-on experience with Snowflake at the administrator level, including RBAC, warehouse sizing and cost management, masking and row-access policies, replication, secure data sharing, and performance tuning.

• Proficient in Python and SQL.

• Practical experience with Prefect, Airflow, or Dagster, including running and upgrading on Kubernetes or similar infrastructure.

• Hands-on expertise with Terraform on AWS, including EKS or ECS, IAM, VPC networking, S3, Lambda, and secrets management.

• Experience with dbt at scale, including project structure, macros, testing, and consolidating overlapping projects.

• Ownership of CI/CD processes for data systems, preferably using GitHub Actions, covering keyless OIDC authentication, multi-environment promotion, and automated quality gates.

• Familiarity with managed ingestion tools like Fivetran, CDC, Snowpipe, and event or behavioral data platforms such as Amplitude or Segment.

• Operational discipline with observability tools like Datadog, alerting design, incident response, and follow-up after incidents.

• Proven experience in platform migration or consolidation, including parity validation and cutover planning.

• Experience in setting standards that are adopted by other engineers through reusable frameworks, reference implementations, documentation, and code review.

• Ability to work effectively with offshore and partner delivery teams.

• Excellent written communication skills, especially in design documents, runbooks, and incident write-ups.

• Practical experience in data governance is preferred, including PII classification, access controls, retention, and lineage.

• Exposure to ML or AI workloads is a plus.

• Experience in sectors such as media, sports, entertainment, live events, ticketing, or consumer and fan data environments is preferred.

🏝️ Benefits

• Short- and long-term incentives, as applicable.

• Opportunities for growth and development.

• Health care coverage.

• Retirement benefits.

• Vacation and additional paid time off.

• Other benefits and offerings.

• Reasonable accommodations for qualified individuals with disabilities.