Databricks Developer

This is a fully remote position, open to applicants in Poland, +4 more countries.

📋 Description

• Design, develop, and sustain scalable data ingestion, transformation, and analytics pipelines utilizing Databricks.

• Create data processing solutions employing Apache Spark and PySpark.

• Handle large-scale datasets while implementing performance tuning and optimization strategies.

• Architect and uphold Delta Lake / Delta Table solutions to facilitate reliable and scalable data processing.

• Develop and manage unit tests to guarantee the quality and reliability of Databricks workloads.

• Implement and oversee Databricks Workflows for data processing and pipeline orchestration.

• Work with columnar storage formats such as Parquet and optimize data processing and storage solutions.

• Assess business and technical requirements and convert them into effective data engineering solutions.

• Collaborate with cross-functional teams to ensure data pipelines and solutions align with technical and business necessities.

• Apply best practices in data engineering and software development to provide maintainable, scalable, and reliable solutions.

⛳️ Requirements

• Over 5 years of practical software development experience.

• Extensive expertise in Apache Spark, including hands-on experience delivering solutions on Databricks.

• Advanced Python and data engineering capabilities, with significant experience in PySpark, Pandas, and related libraries.

• Experience in developing and sustaining unit tests for Databricks workloads.

• Strong understanding of columnar storage formats, such as Parquet.

• Practical experience with Delta Lake and Delta Tables.

• Proven background in working with small to large datasets, including performance tuning and optimization.

• Experience in building data ingestion, transformation, and analytics pipelines.

• Hands-on experience with Databricks Workflows.

• Strong analytical and problem-solving abilities with meticulous attention to detail.

• Conversational English — B2 level or higher.

• Conversational Ukrainian.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Flexible working hours and remote work options.

• Opportunities for professional development and training.

• Access to the latest tools and technologies.

• Collaborative and innovative team environment.