This is a fully remote position, open to applicants in Europe.

📋 Description

• Develop a reusable Python ingestion framework to expedite customer integrations and replace the outdated Java ETL.

• Maintain synchronization between PostgreSQL and Snowflake through dependable real-time streaming and change data capture pipelines.

• Leverage LLMs to correlate customer schemas and minimize manual integration efforts.

• Manage data pipelines in production, which includes deployment, monitoring, ensuring data quality, recovery, and secure handling of customer information.

• Enhance existing code and design new infrastructure as a member of the Data & AI Platform team.

⛳️ Requirements

• Over 4 years of experience in data engineering.

• Proven experience in constructing Python and SQL pipelines for complex integrations involving unfamiliar data sources, inconsistent schemas, and incomplete documentation.

• Practical experience with streaming and change data capture, encompassing schema adjustments, backfills, and recovery following failures.

• Production experience with PostgreSQL, Snowflake, cloud infrastructure (preferably AWS), and Airflow or similar tools.

• Demonstrated ability to manage deployment, monitoring, and troubleshooting processes.

• Regular use of AI tools, with specific examples of integrating them into workflows to automate tasks and enhance development.

• Familiarity with dbt and DuckDB.

• Experience in managing infrastructure with Terraform.

• Understanding of cross-warehouse data sharing or LLM-assisted data integration.

• Eligibility to work in the location applied for.