Remotery

Senior Lead Data Engineer, Content Engineering

atParamountUS flagNew YorkFull-timeUncategorizedSenior$156.8k – $235.2k/year

Posted 11 hours ago

This is a fully remote position, open to applicants in New York.

📋 Description

• Construct and Manage Large-Scale Feature Pipelines: Create and oversee batch/streaming pipelines (Spark, Flink, Databricks, Airflow) that generate ML features for ranking models.

• Guarantee Point-in-Time Accuracy: Create feature sets that support unbiased offline training and reliable online inference.

• Build Embedding & Content Pipelines: Develop scalable workflows for metadata, imagery, and multimodal representations; collaborate with Science teams to implement new models.

• Design Data Foundations: Create Delta/Parquet data models and medallion layers, optimizing storage layout and partitioning for latency and cost efficiency.

• Real-Time Engineering: Develop Kafka-based systems for real-time features and user-activity aggregations, ensuring effective handling of out-of-order events and exactly-once semantics.

• Governance & Leadership: Establish data quality standards and schema evolution processes while working across ML pods to convert model requirements into infrastructure.


⛳️ Requirements

• Over 7 years of experience in large-scale data or software engineering.

• Extensive expertise in Spark (PySpark/Scala), Databricks, Airflow, and Kafka.

• Strong proficiency in feature pipelines, temporal joins, and addressing training-serving discrepancies.

• Familiarity with AWS/Azure/GCP and high-performance engines such as Snowflake or Redshift.

• Skilled in programming with Python and SQL emphasizing performance optimization.

• Experience in personalization fields (search, ranking, or recommendation systems).

• Background in supporting petabyte-scale data lakehouses or feature stores.

• Understanding of GenAI/RAG systems, multimodal content, or Delta Live Tables.

• Knowledge in Causal Inference, experimentation signals, or ML evaluation workflows.

• Proficient in using Terraform for managed, repeatable deployments.


🏝️ Benefits

• Medical

• Dental

• Vision

• 401(k) plan

• Life insurance coverage

• Disability benefits

• Tuition assistance program

• PTO

People also viewed

Anchor Utility11 hours ago

Rate Analyst

US flagTexas OnlyFull-timeUncategorized
ApplyView job
Honeywell11 hours ago

HSE Manager

US flagNorth Carolina OnlyFull-timeUncategorized
ApplyView job
Cision France11 hours ago

People Partner

CA flagCanada OnlyFull-timeUncategorized$85k/year
ApplyView job
Navigate Power11 hours ago

B2B Outside Sales Consultant

US flagPennsylvania OnlyFreelanceUncategorized$50k – $250k/year
ApplyView job
TELUS11 hours ago

Business Development Executive, Early Career – European Language Required

GB flagUnited Kingdom OnlyFull-timeUncategorized
ApplyView job
Gilead Sciences11 hours ago

Statistical Programmer II

US flagUnited States OnlyFull-timeUncategorized$107.2k – $138.7k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers