Data Engineering Lead

This is a fully remote position, open to applicants in United States.

📋 Description

• Establish the technical trajectory for Seer's data platform — outline the architecture, standards, and roadmap for other engineers to follow.

• Design Seer’s data platform comprehensively, utilizing BigQuery, Dataflow, Airflow, Cloud Run, Cloud Functions, and Cloud Storage while balancing cost, latency, and freshness to ensure scalable solutions without surprises.

• Define the standards for data modeling and serving, including dbt project structure, LookML, Looker explores, and embedded Looker experiences within client portals and internal tools.

• Guide data engineers and analysts throughout the team — collaborate on complex challenges, conduct architectural reviews, provide constructive feedback on PRs to enhance skills (beyond just code), and facilitate workshops on TDD, git internals, or LLM-augmented engineering.

• Advocate for engineering best practices — implement TDD, modular Pytest suites, clean Git workflows, and CI/CD pipelines with multi-stage deployments. Set visible standards and assist the team in achieving them.

• Optimize expensive processes: identify slow queries, redundant DAGs, and ineffective storage policies. Reduce BigQuery expenses and dashboard load times while bringing the team along on this journey.

• Utilize tools like ChatGPT, Claude, Gemini, and prompt-evaluation harnesses to enhance research, code review, and pipeline development — creating AI-driven workflows that boost Seer’s engineering productivity.

• Document processes and educate others. Develop playbooks that withstand system changes, decision records that clarify *why* decisions were made (not just *what*), and onboarding paths that benefit new hires.

⛳️ Requirements

• Proven experience in leading through technical expertise and mentorship — setting high standards within a data engineering team, enhancing the capabilities of engineers, and gaining trust without relying on formal authority.

• In-depth knowledge of the GCP data stack — BigQuery (including partitioning, clustering, materialized views, cost optimization), Dataflow, Airflow, Cloud Run, Cloud Functions, Datastore, and Cloud Storage — with a successful history of architecting production pipelines from start to finish.

• Mastery of the analytics and BI layer — modular dbt projects, reusable LookML, Looker explores and embeds, and Looker Studio dashboards that integrate CRM and ad data for comprehensive attribution. You write clean SQL and can refactor complex SQL into more efficient versions.

• Proficient in Python — writing idiomatic, performant, and well-tested code. You develop modular Pytest suites with custom plugins, adopt TDD as a standard practice, and have introduced contract or snapshot testing to catch potential issues not addressed by unit tests.

• Advanced Git and version control skills to assist teammates effectively. You have created CI/CD pipelines with multi-stage deployments, automated testing gates, and maintained a disciplined approach to production debugging.

• Familiarity with AI and LLM tools that goes beyond basic usage — crafting nuanced, context-aware prompts, building evaluation harnesses for comparing phrasing and sampling strategies, and integrating LLMs into engineering workflows to maximize their value.

• Strong project management skills — adept at Agile ceremonies that provide real value, proficient in Wrike (or similar) for tracking dependencies and prioritization, and possessing the discernment to modify processes when they no longer benefit the team.

• Versatile communication skills — capable of informing an SLT stakeholder about a cost-saving initiative and guiding a junior engineer through a complex rebase within the same day, ensuring clarity for both parties.

🏝️ Benefits

• The salary range for this position is $120,000-130,000.

• Your final compensation will be based on your skills and experience.

• Compensation evaluations will occur at least annually.

• Highlights of benefits offered.