Senior Data Engineer

This is a fully remote position, open to applicants in Nebraska, +4 more states.

📋 Description

• Take charge of transforming raw transactional and event data into reliable, well-structured datasets that drive BetMGM's analytics, machine learning, and operational systems.

• Create, develop, and manage batch, micro-batch, and streaming pipelines that feed into Snowflake — using Prefect for orchestration on ECS Fargate, with dbt for data transformation, and Snowpipe Streaming and Kafka for event ingestion.

• Oversee the complete dbt lifecycle (from sources to staging, intermediate, and marts) incorporating model contracts, freshness service level agreements (SLAs), automated testing, and version-controlled documentation.

• Establish Snowflake objects (warehouses, role-based access control, resource monitors, Dynamic Tables, Iceberg tables) using Terraform — ensuring no ClickOps in production environments.

• Develop AWS-native infrastructure for data workloads, including S3, ECS Fargate, Lambda, EMR Serverless, Glue Catalog, IAM, Secrets Manager, and VPC endpoints — all managed through Terraform.

• Maintain continuous integration and continuous deployment (CI/CD) pipelines (using GitLab CI or GitHub Actions) that enforce every change with linting, dbt builds, unit testing, contract validations, and AI-assisted code reviews.

• Optimize warehouse sizing, clustering, and query patterns for cost efficiency and latency; track credit usage via ACCOUNT_USAGE; properly size before scaling up.

• Design role-based access control, masking policies, and row-access policies that comply with regulatory requirements while avoiding access bottlenecks.

• Manage freshness SLAs and data contracts for the gold layer; set up Monte Carlo coverage for volume, freshness, schema, and distribution; handle incidents from start to finish.

• Collaborate with analytics engineers, data scientists, and machine learning platform engineers on shared standards (including naming conventions, testing protocols, observability, data lineage, and cost attribution).

⛳️ Requirements

• Bachelor’s or Master’s degree in Computer Science, Statistics, Mathematics, or another STEM field — or equivalent practical experience.

• Over 5 years of experience in building production data pipelines using a modern tech stack (Python, SQL, dbt, and cloud services).

• Extensive experience with Snowflake, including advanced administration skills: warehouse sizing, RBAC, resource monitoring, Streams/Tasks, Dynamic Tables, secure data sharing, and cost optimization using ACCOUNT_USAGE.

• Strong knowledge of AWS services — including S3, ECS/Fargate, Lambda, IAM, Secrets Manager, and VPC — along with production experience in at least one of EMR Serverless, Glue, or MWAA.

• Proficient in Terraform for both cloud infrastructure and Snowflake — you have taken ownership of infrastructure as code (IaC), not just interacted with it.

• Familiarity with orchestration tools — such as Prefect, Airflow, or Dagster — with a clear understanding of when to utilize each tool effectively.

• Experience in CI/CD practices — you have developed quality gates that prevent the introduction of faulty code, rather than merely creating YAML pipelines that pass.

• Results-oriented mindset — you articulate previous work in terms of SLAs, incidents handled, and customers served, rather than just listing tools used.

🏝️ Benefits

• Medical, Dental, Vision, Life, and Disability Insurance

• 401(k) plan with company matching

• Pre-tax spending accounts, including health care FSA and commuter savings options

• Flexible paid time off policy

• Professional development reimbursement and ongoing skills training opportunities

• Access to employee resource groups

• Swag, ticket giveaways, and additional perks!