Data Engineer – Databricks, PySpark

This is a fully remote position, open to applicants in Poland.

📋 Description

• Design and sustain data pipelines for processing substantial volumes of data in both batch and near real-time scenarios.

• Utilize Databricks, PySpark, and SQL to transform, validate, and enhance data.

• Assist in the implementation of Data Lakehouse solutions while adhering to engineering standards and best practices.

• Construct, test, and automate ETL processes and data workflows.

• Elevate data quality, organization, and reliability across various platforms.

• Collaborate with engineers, analysts, and stakeholders to provide scalable solutions.

• Diagnose, optimize, and oversee data pipelines.

• Keep abreast of contemporary data engineering technologies.

• Ensure that deliverables conform to quality standards and project specifications.

⛳️ Requirements

• A minimum of 2 years of experience in Big Data development and database design.

• Strong practical experience with SQL.

• Proficient in Python development.

• Hands-on experience with PySpark.

• Experience in developing and maintaining data pipelines using Databricks.

• Familiarity with Delta Lake.

• Knowledge of Medallion Architecture (Bronze, Silver, Gold layers).

• Strong analytical abilities and problem-solving skills.

• Intermediate or higher proficiency in English.

• Experience with AWS services, including S3, is a plus.

🏝️ Benefits

• Employees have the flexibility to work remotely.