Staff Data Engineer

This is a fully remote position, open to applicants in California.

📋 Description

• Design and manage a scalable identity resolution platform.

• Develop pipelines and services for ingesting, normalizing, linking, and versioning identity data across various sources.

• Ensure that the matching logic is both deterministic and probabilistic, providing transparency, auditability, and measurability.

• Collaborate with product and analytics teams to present identity data through dependable, well-documented APIs and datasets.

• Construct and maintain both batch and streaming pipelines utilizing modern data stack tools.

• Produce clear documentation, standards, and runbooks for identity and governance systems.

• Oversee foundational data governance aspects including data lineage, quality checks, schema enforcement, and access controls.

• Apply privacy-by-design principles (handling PII, enforcing consent, and establishing retention policies).

• Work alongside legal, privacy, and security teams to implement regulatory requirements (such as GDPR and CCPA).

• Set up monitoring and alerting systems for data quality, freshness, and integrity.

⛳️ Requirements

• Experience in production data engineering.

• Bachelor's degree in computer science, a related field, or equivalent experience.

• Proficient in Spark and Scala, with a proven track record in building data infrastructure in Spark using Scala.

• Experience in executing significant technical initiatives and developing reliable, large-scale services.

• Background in delivering APIs supported by relationship-intensive datasets.

• Familiarity with implementing data governance practices, including data quality, metadata management, and access controls.

• Strong grasp of privacy-by-design principles and the management of sensitive or regulated data.

• Knowledge of data lakes, cloud warehouses, and various storage formats.

• High proficiency with AWS services.

• Exceptional written and verbal communication skills.

• Proven success in designing and implementing scalable and efficient data infrastructure.

• Attentive to detail in the implementation of automated data quality checks.

• Strong collaborative skills with cross-functional teams.

• Demonstrated ability to leverage AI to enhance speed and quality in daily workflows for relevant outputs.

• A solid track record of critically evaluating and verifying AI-assisted work (including testing, source-checking, data validation, and peer review).

• High integrity and ownership: you safeguard sensitive data, avoid excessive reliance on AI, and remain accountable for final decisions and deliverables.

🏝️ Benefits

• Health insurance.

• Equity opportunities.

• Flexible work arrangements.

• Professional development.