Data Engineering Lead

📋 Description

• Establish the technical vision for Seer’s data platform — outline the architecture, standards, and roadmap that other engineers will adhere to.

• Design the complete architecture of Seer’s data platform utilizing BigQuery, Dataflow, Airflow, Cloud Run, Cloud Functions, and Cloud Storage, while balancing cost, latency, and freshness to ensure scalable solutions without unexpected challenges.

• Define the standards for modeling and serving data, from dbt project organization to LookML, Looker explores, and integrated Looker experiences within client portals and internal tools.

• Guide and mentor data engineers and analysts across the team — collaborate on complex issues, conduct architectural reviews, provide constructive feedback on pull requests, and occasionally lead workshops on TDD, git fundamentals, or LLM-enhanced engineering practices.

• Advocate for engineering best practices — TDD, modular Pytest suites, clean Git workflows, and CI/CD pipelines with multi-stage deployments. Set clear expectations and assist everyone in achieving them.

• Optimize costly processes: identify slow queries, redundant DAGs, and ineffective storage policies. Reduce BigQuery expenses and enhance dashboard performance, while bringing the team along in the process.

• Utilize tools such as ChatGPT, Claude, Gemini, and prompt-evaluation frameworks to expedite research, code reviews, and pipeline development — and create AI-driven workflows that enhance Seer’s engineering productivity.

• Document and instruct. Develop playbooks that endure system changes, decision records that clarify *why* decisions were made, and onboarding resources that benefit incoming team members.

⛳️ Requirements

• Proven experience in leading through technical expertise and mentorship — establishing high standards on a data engineering team, elevating the capabilities of your peers, and gaining trust without relying on formal authority.

• Mastery of the GCP data stack — BigQuery (including partitioning, clustering, materialized views, cost optimization), Dataflow, Airflow, Cloud Run, Cloud Functions, Datastore, and Cloud Storage — with a history of architecting production pipelines from start to finish.

• In-depth knowledge of the analytics and BI layer — modular dbt projects, reusable LookML, Looker explores and embeds, and Looker Studio dashboards that integrate CRM and ad data for comprehensive attribution. You excel at writing clean SQL and improving complex SQL queries.

• Advanced Python proficiency — idiomatic, efficient, and thoroughly tested. You create modular Pytest suites with custom plugins, adopt TDD as a standard practice, and have implemented contract or snapshot testing to capture what unit tests might overlook.

• Strong Git and version control skills used to assist teammates. You have designed CI/CD pipelines that feature multi-stage deployments, automated testing gates, and disciplined production debugging.

• Proficiency in utilizing AI and LLM tools beyond basic functions — crafting nuanced, context-sensitive prompts, developing evaluation frameworks to assess phrasing and sampling methods, and integrating LLMs into engineering workflows where they provide tangible benefits.

• Solid project management skills — familiarity with Agile practices that deliver value, proficiency in Wrike (or similar tools) for tracking dependencies and prioritization, and the adaptability to modify processes when they no longer serve the team's needs.

• Versatile communication skills — capable of briefing a senior leadership team member on a cost-saving project and explaining a complex rebase to a junior engineer within the same day, ensuring both leave with a clear understanding.

🏝️ Benefits

• The salary range for this position is $120,000-130,000.

• Your final compensation will be determined based on your skills and experience.

• Compensation evaluations will occur at least once a year.

• Highlights of benefits offered.