Senior Data Engineer, Ads Data – AI Native

This is a fully remote position, open to applicants in United States, +1 more country.

📋 Description

• Enhance, expand, and operate batch and streaming pipelines along with Delta Lake tables on Databricks.

• Advance the platform in terms of ingestion, orchestration, storage, and serving as the volume of data and business requirements increase.

• Collaborate with business analysts, ad strategists, and sales teams to transform business inquiries into sustainable data and insights products.

• Develop and maintain dbt models and tests for Silver/Gold datasets that are curated.

• Take ownership of the semantic layer and core business metrics.

• Manage Databricks and AWS infrastructure using code through Terraform.

• Integrate and reconcile data from advertising platforms, CRM systems, partner feeds, and internal databases.

• Transition away from outdated tools and reports.

• Implement data quality checks, monitoring, lineage, SLAs, and alert systems.

• Establish data contracts with upstream data producers and lead root-cause analyses when data issues arise.

• Optimize Spark and Databricks workloads for performance and cost efficiency.

• Set up version control, CI/CD, code review, testing, and documentation best practices.

• Mentor engineers and contribute to the development of the team’s technical roadmap.

• Enforce data governance, privacy, and access controls across the platform.

• Develop LLM tools, skills, and agents to enhance debugging, development, and system maintenance.

• Become the primary engineer for foundational data models and pipelines within the first year.

• Minimize pipeline runtime and compute expenditures while delivering documented, reliable datasets and metrics.

⛳️ Requirements

• Over 5 years of experience in data engineering, constructing production data platforms at an enterprise scale.

• Proven ownership of data products and experience in cross-functional collaboration.

• Extensive, hands-on expertise with Apache Spark, including PySpark and/or Scala, partitioning, shuffle behavior, joins, skew management, caching, and job optimization.

• Profound knowledge of Databricks, encompassing Delta Lake, Unity Catalog, Jobs/Workflows, Databricks Asset Bundles, Structured Streaming, and SQL Warehouses.

• Experience in optimizing workloads for both performance and cost.

• Strong skills in SQL and Python or Scala.

• Practical experience with dbt, including model design, testing, documentation, and incremental strategies.

• Hands-on experience with Terraform and AWS, covering S3, IAM, and networking fundamentals.

• Familiarity with orchestration frameworks like Airflow or Databricks Workflows.

• Experience with Git, CI/CD, and developing reliable, observable pipelines.

• Understanding of data modeling, including dimensional modeling, star schemas, and medallion architecture.

• Experience in building semantic layers or defining metrics.

• Experience in inheriting, refactoring, and enhancing existing data models and pipelines.

• Familiarity with AI/LLM coding assistants such as GitHub Copilot, Cursor, or Claude Code.

• Strong business insight along with excellent communication abilities.

• Knowledge of GDPR, CCPA, SOX, and familiarity with BI tools such as Domo, Tableau, or Looker.

• Bachelor's degree or equivalent experience in Computer Science, Engineering, or a related discipline.

• Regular professional use of AI coding assistants.

• Ability to review, test, and take ownership of AI-generated code.

• Capability to keep up with rapidly evolving AI tools.

• Experience in advertising, ad tech, or media data is preferred.

🏝️ Benefits

• Comprehensive medical, dental, vision, life, and disability insurance plans (fully covered for US employees).

• Life and disability insurance, along with supplemental medical and dental plans for Canadian employees.

• 401(k) plan with a company matching program available in the US.

• RRSP with DPSP plan for employees in Canada.

• Paid parental leave.

• Mental Wellness Program & Employee Assistance Program (EAP) for mental health support.

• Flexible PTO.

• Company-wide holidays throughout the year.

• Summer and winter shutdowns.

• Learning & Development opportunities.

• Provision of equipment, tools, and reimbursement support to facilitate a productive remote work environment.

• Complimentary Life360 Platinum Membership and Tile trackers.

• Equity compensation in the form of Restricted Stock Units (RSUs).