Senior Data Platform Engineer, Snowflake

This is a fully remote position, open to applicants in Poland.

📋 Description

• Develop data ingestion pipelines that integrate AI tools and internal systems into Snowflake.

• Enhance and fortify the existing Snowflake infrastructure, which consists of schemas and tables that have evolved organically without the input of data engineering, bringing them up to industry standards.

• Implement deployments through CI/CD pipelines into Airflow or AWS Glue.

• Oversee and process access requests.

• Work collaboratively with product managers and engineers to determine data requirements.

⛳️ Requirements

• Over 5 years of experience in data engineering.

• Proficiency in Python for scripting, API development, and pipeline construction.

• Familiarity with Apache Airflow for pipeline orchestration; Dagster or Prefect can be considered as alternatives.

• Experience with AWS services, particularly Glue and Lambda, along with a background in deploying and maintaining production workloads.

• Knowledge of Apache Spark for distributed processing, especially within AWS Glue.

• Preference for Snowflake as a data warehouse; familiarity with Redshift or BigQuery is acceptable if concepts can be easily adapted.

• Experience with CI/CD pipelines, such as GitHub Actions or similar, as this is the method for deploying pipelines and scripts to Airflow and Glue.

• API experience, including consuming third-party APIs and developing internal APIs using Python.

• Proficient with Git/GitHub for version control, branching strategies, and pull request workflows.

• Knowledge of PostgreSQL or other OLTP databases for operational data access and integration.

🏝️ Benefits

• Fully remote work environment.

• Health insurance (specific details not specified).

• Opportunities for professional development.