Middle/Senior Data Engineer

This is a fully remote position, open to applicants in Poland.

📋 Description

• Create, implement, and sustain data pipelines and governance frameworks using Databricks (Unity Catalog, Delta Lake, Workflows)

• Perform comprehensive data analysis, profiling, and processing to categorize data by domain and identify personal data (PII)

• Assist in designing data access systems with an emphasis on data contracts, catalog access, and automation of grant/rollback processes managed as Infrastructure-as-Code using Terraform

• Evaluate access architectures and contracts to guarantee flexibility and reusability

• Ensure that data products and access controls are deployed in accordance with contracts and function without unintended consequences

• Maintain thorough documentation for data contracts, access models, and governance procedures

• Oversee and assist deployed pipelines and quality checks to confirm they align with data quality and performance standards

• Proactively gather information and internal solutions to promote reusability and adoption of common data platform technologies

⛳️ Requirements

• Over 3 years of experience in Data Engineering

• Strong proficiency in Python

• Experience with Spark/PySpark for streaming, batch, and asynchronous data processing

• Expertise in data modeling

• Knowledge of distributed data processing

• Proficient in SQL

• Familiarity with data warehouse/lakehouse architecture

• Practical experience with Databricks, GCP, or AWS

• Proficient in Terraform

• Experience in designing data access and governance solutions

• Understanding of data products and governance controls

🏝️ Benefits

• Health insurance

• Flexible work arrangements

• Opportunities for professional development