Data Engineer

This is a fully remote position, open to applicants in Argentina.

📋 Description

• Take ownership of and enhance data ingestion pipelines from a variety of external and internal sources, encompassing ingestion, cleansing, curation, entity resolution, and event-driven publishing.

• Design, construct, and sustain scalable batch and event-driven data workflows utilizing Python, SQL, and AWS.

• Address challenges related to identity resolution, entity matching, and deduplication across diverse data sources.

• Develop and uphold frameworks for data quality and validation, including monitoring for freshness, null rates, consistency checks, and schema drift detection.

• Oversee ingestion-boundary data, proactively identifying, troubleshooting, and resolving issues.

• Collaborate with event infrastructure and downstream identity-service teams to track data and events throughout the entire process.

• Work alongside Product, Assessments, Analytics, and Engineering teams to ensure pipelines align with product and business needs.

• Construct and manage data infrastructure using AWS Glue, Athena, S3, and DynamoDB.

• Automate infrastructure and pipeline modifications utilizing Terraform or AWS CDK.

• Engage in production incident responses and execute remediation measures.

• Enhance the reliability, observability, scalability, and maintainability of data pipelines and platform infrastructure.

⛳️ Requirements

• A minimum of 5 years of experience in developing and managing production-grade data pipelines and data infrastructure.

• Extensive experience with both batch processing and event-driven architectures.

• Practical experience with AWS Glue, Athena, and S3-based data lakes, including layered or medallion-style data transformations.

• Strong skills in Python and SQL for data processing, transformation, and analysis.

• Proven experience addressing identity resolution, entity matching, and deduplication challenges, employing both exact and fuzzy matching techniques.

• Understanding of durable identifiers and identity-mapping strategies, including the ability to maintain consistent first-seen or locked mappings over time.

• Familiarity with event schemas and schema-registry-backed contracts, such as Protobuf, along with an understanding of how schema evolution affects downstream consumers.

• Excellent troubleshooting skills and ability to respond to production incidents effectively.

• Capability to comprehend and debug code written in functional or concurrent programming languages, such as Elixir.

• Strong communication and collaboration abilities across engineering, product, analytics, and other technical teams.

• Experience with Elixir/Phoenix or other BEAM-based concurrent processing frameworks.

• Familiarity with DynamoDB-backed identity, lookup, or matching services.

• Experience with the Snowflake ecosystem, including data modeling, Snowpipe, Streams, and Tasks.

• Proven experience in implementing data observability, including alerts for freshness and staleness, null-rate monitoring, schema drift detection, and data-quality dashboards.

• Experience in managing infrastructure through Terraform or AWS CDK.

🏝️ Benefits

• Full-time contract employment.

• Remote work arrangement.