Senior Data Engineer – Data Platform

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of production data systems and domains from the point of ingestion to delivery.

• Create ingestion patterns that effectively manage incomplete, duplicated, delayed, malformed, and evolving partner data.

• Establish contracts among source systems, ingestion processes, transformation, and downstream consumers.

• Develop data models that include clear granularity, keys, history, and definitions.

• Safely evolve schemas, pipelines, and datasets while implementing retries, backfills, and replays to prevent corruption or duplication.

• Enhance pipeline performance, observability, and failure management.

• Collaborate with Engineering, Product, Operations, and Analytics to identify data requirements.

• Make architectural decisions and implement changes to production systems safely.

• Assist other engineers in making informed decisions regarding data infrastructure.

• Undertake additional responsibilities as required.

⛳️ Requirements

• A minimum of a Bachelor's degree.

• Over 5 years of experience in building production data systems.

• Strong foundations in SQL and data modeling, including aspects like granularity, keys, history, dimensional modeling, and the trade-offs of various data representations.

• Experience in designing and managing batch and asynchronous data pipelines, including understanding the failure modes in distributed systems.

• Familiarity with cloud data platforms, alongside knowledge of storage, query performance, and cost implications.

• Strong instincts regarding data integrity, encompassing validation, idempotency, late and duplicated records, schema modifications, backfills, and reprocessing.

• Proficient in Python for developing pipelines, services, tooling, and tests.

• Capability to modify production systems in an orderly and reversible manner without disrupting their consumers.

• Excellent written communication skills, including the ability to document technical decisions and convey changes to non-engineering stakeholders.

• Applicants must possess authorization to work in the U.S.; Vida does not sponsor work visas.

• Employees must live in or be able to work from the U.S.; international work is not permitted.

• Previous healthcare experience or familiarity with every technology used by Vida is not required.

🏝️ Benefits

• An Equal Employment Opportunity and Affirmative Action employer.

• Reasonable accommodations for qualified individuals with disabilities and disabled veterans during the job application process.