Senior Data Engineer

This is a fully remote position, open to applicants in Argentina.

📋 Description

• Take full ownership of Fundable’s data platform, overseeing everything from architecture to production.

• Develop concurrent ingestion processes for provider APIs, webhooks, files, and web scraping, implementing rate limits and failure management.

• Coordinate scheduled workflows and historical backfills utilizing Airflow, Dagster, or Prefect.

• Create event-driven intake systems with Pub/Sub or Cloud Tasks, incorporating retries, dead-letter queues, and controlled replay mechanisms for failed tasks.

• Preserve the original provider payloads in Google Cloud Storage prior to transforming them using dbt.

• Establish medallion layers for raw, standardized, and curated datasets.

• Implement idempotent upserts, secure retries, replay capabilities, and historical backfills.

• Ensure data traceability to its source and guarantee safe reprocessing.

• Define data contracts along with automated validation, pipeline testing, freshness, and failure monitoring.

• Design and enhance PostgreSQL schemas, queries, and loading methodologies.

• Integrate provider enrichment and LLM-based structured extraction from textual data.

⛳️ Requirements

• Proven experience in building multi-source production data platforms from inception to deployment, including initial architecture and pipeline implementation.

• Practical experience with workflow orchestration tools like Airflow, Dagster, Prefect, or Temporal, including handling dependencies, retries, and failure recovery; Airflow expertise is preferred.

• Hands-on experience with dbt and Google Cloud Platform (GCP).

• Background in developing event-driven systems utilizing messaging or task queues, focusing on asynchronous processing and failure recovery.

• Strong proficiency in SQL and PostgreSQL, encompassing data modeling, query optimization, and dependable incremental loading.

• Familiarity with data contracts, raw and curated storage solutions, retention policies, and automated quality assurance checks.

• Solid Python programming skills, including asynchronous HTTP and concurrent processing with tools like asyncio, aiohttp, or HTTPX.

• Practical experience in web scraping using tools such as Scrapy, Playwright, Selenium, or Beautiful Soup, with a focus on concurrent scraping, rate limits, and failure management.

• Must be physically located in Argentina.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Opportunities for professional development and career advancement.

• Flexible working hours and remote work options.

• Collaborative and inclusive work environment.