
Senior Data Engineer
Posted 1 day ago

Posted 1 day ago
This is a fully remote position, open to applicants in New York.
• Act as the technical lead for data pipelines that support various measurement or optimization use cases.
• Design and construct scalable ETL/ELT pipelines that ingest and standardize data from media, commerce, engagement, and partner sources.
• Create and manage extensive identity graph infrastructure, which includes entity resolution, match/merge logic, and graph storage.
• Collaborate with the Identity and Privacy teams to effectively utilize Real ID and clean-room infrastructure, ensuring compliance with consent-aware and privacy-safe standards.
• Develop integration strategies with walled gardens, publishers, and retail media partners to incorporate exposure and outcome data.
• Work alongside Data Science teams to establish feature pipelines and enhance measurement and optimization use cases.
• Implement frameworks for data quality, lineage, and monitoring tailored for high-volume, high-cardinality data.
• Lead architecture evaluations, establish engineering standards, and provide mentorship to data engineers.
• Foster ongoing enhancements in pipeline reliability, scalability, and cost-effectiveness.
• Bachelor's degree in Computer Science, Engineering, or a related discipline.
• Over 5 years of experience in data engineering, with a focus on large-scale pipeline or data platform architecture ownership.
• Proven track record in designing and managing large-scale, distributed data pipelines.
• Strong expertise in SQL and proficiency in at least one programming language (e.g., Python, Scala, Java).
• Experience with prominent cloud/data platforms (e.g., Snowflake, Databricks, AWS) and orchestration tools (e.g., Airflow).
• Practical experience with entity resolution, identity graphs, or large-scale record linkage, including familiarity with graph databases or processing frameworks (e.g., Neo4j, TigerGraph, or similar).
• Understanding of privacy-safe data engineering practices and clean room environments.
• Excellent communication and stakeholder management skills, capable of articulating architectural decisions to both technical and business audiences.
• Experience in transforming data pipelines from custom or professional-services applications into standardized, repeatable data products.
• Hands-on experience within AdTech/MarTech or data-driven product ecosystems.
• Familiarity with partner/API integration patterns for walled gardens, DSPs, SSPs, or retail media networks (e.g., Meta, Amazon Marketing Cloud (AMC), Google, The Trade Desk).
• Experience with streaming data platforms (e.g., Kafka, Kinesis).
• Familiarity with clean room platforms (e.g., Snowflake Data Clean Rooms, LiveRamp, InfoSum, or similar).
• Full-stack development skills and experience in designing and securing APIs (REST/GraphQL) that expose data products to internal systems, partners, and AI agents.
• Exposure to LLMs, RAG, MCP, and AI coding assistants (e.g., Claude Code, GitHub Copilot), with a solid understanding of validating AI-generated code and managing agentic or AI-assisted workflows.
• Competitive salary and performance-based bonuses.
• Comprehensive health, dental, and vision insurance.
• Flexible work hours and remote work options.
• Professional development opportunities and support for continuing education.
• A collaborative and innovative work environment.
Highmark Health
Prompt Therapy Solutions Inc
Jones Lang LaSalle Americas, Inc.
Allshore Talent
Get handpicked remote jobs straight to your inbox weekly.