Senior Data Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Take responsibility for data ingestion from transactional databases through CDC, SaaS sources, application events, and external APIs.

• Manage backfills, retries, schema modifications, and late-arriving data.

• Oversee the dbt project from start to finish, covering staging to marts, as well as tests, documentation, incremental models, naming conventions, and contracts.

• Enhance BigQuery performance by utilizing partitioning, clustering, query optimization, and cost monitoring.

• Deliver low-latency analytics from an OLAP store like ClickHouse through effective table design, sorting keys, materialized views, and aggregation strategies.

• Administer the GCP ecosystem surrounding the warehouse, including Cloud Storage, Cloud Run services, IAM, and data retrieval from external sources.

• Execute production Airflow scheduling, manage dependencies, handle retries, alerting, and oversee operational processes.

• Establish freshness and quality checks, addressing root causes effectively.

• Implement CI/CD practices and tests for data-related code.

• Identify and implement beneficial LLM and agent applications for data discovery, quality monitoring, and troubleshooting.

• Conduct code reviews, mentor junior and mid-level engineers, and keep thorough documentation of decisions and runbooks.

• Collaborate with engineering, product, design, and operations teams.

⛳️ Requirements

• 5+ years of experience in data or backend engineering, primarily in production environments.

• Proficient in Python, with extensive experience in writing and debugging production pipelines and services.

• Advanced SQL skills, including window functions, complex transformations, and understanding query performance issues.

• 2+ years of hands-on experience with production dbt, encompassing incremental models, macros, tests, and documentation.

• Experience managing production Airflow DAGs, including backfills, monitoring, and troubleshooting issues.

• Strong BigQuery expertise, including modeling, partitioning, clustering, and query and cost optimization.

• Good familiarity with GCP, covering Cloud Storage, IAM, and general cloud data infrastructure.

• Solid experience with PostgreSQL, focusing on indexing, query performance, and incremental extraction.

• Experience with stream or distributed processing frameworks, such as Apache Flink or similar technologies.

• Comfortable working in a Linux VM environment, including Bash, cron, Docker, log analysis, SSH, and basic networking.

• Strong engineering practices utilizing Git, CI/CD, tests, and code review, ensuring maintainable code.

• Nice to have: familiarity with ClickHouse or production OLAP, managed ETL/ELT platforms like Airbyte or Fivetran, Datastream, Kafka, Debezium, Kubernetes, Terraform, Tableau or Power BI, LLM APIs, RAG, agent frameworks, and data observability tools, as well as AWS.

• A degree in Computer Science or engineering is a plus, but not mandatory.

🏝️ Benefits

• Genuine ownership over products that serve millions of diners and thousands of restaurants worldwide.

• The chance to tackle complex challenges in payments, hospitality, and customer experience.

• A dynamic environment where innovative ideas swiftly come to fruition.

• Close collaboration with seasoned operators, builders, and industry professionals.

• An opportunity to make a significant impact at one of the fastest-growing restaurant technology companies globally.