Junior Data Engineer, Azure Databricks

This is a fully remote position, open to applicants in India, +1 more country.

📋 Description

• Assist in enhancing and maintaining current SQL Server data platforms and ETL solutions.

• Create and manage data pipelines utilizing Azure Databricks.

• Construct and refine data transformations with PySpark and SQL within Databricks.

• Design ETL/ELT pipelines coordinated through Azure Data Factory.

• Merge data from various sources into the data platform and analytical layers.

• Oversee data models and data warehouse frameworks for analytics.

• Guarantee data quality, scalability, and performance in large-scale data processing pipelines.

• Work alongside BI teams to enhance Power BI and reporting platforms.

• Engage in the adoption of modern cloud-based data architectures under the guidance of senior team members.

⛳️ Requirements

• A minimum of 2 years of experience in Data Engineering, Data Warehouse development, or a related field.

• Proficiency with Azure Databricks.

• Experience in creating data pipelines using PySpark and Spark SQL.

• Understanding of distributed data processing and big data principles.

• Strong SQL capabilities and experience with SQL Server relational databases.

• Familiarity with building data pipelines via Azure Data Factory.

• Knowledge of data processing and performance optimization techniques.

• Nice to have: Expertise in Spark optimization methods, including partitioning, caching, and cluster tuning.

• Nice to have: Understanding of Power BI and Databricks Structured Streaming.

• Nice to have: Experience in transitioning traditional ETL processes to cloud-based architectures.

• Nice to have: Familiarity with Delta Lake and Lakehouse architectures.

• Strong analytical and problem-solving abilities.

• Effective communication and collaboration skills.

• Team-oriented with a proactive attitude.

• Willingness to learn and continuously enhance technical knowledge.

• Ability to adapt and thrive in dynamic environments.

🏝️ Benefits

• Comprehensive compensation and benefits package.

• Health insurance.

• Language courses.

• Relocation program.

• Flexible remote work options.

• Opportunities for professional development.

• Certification programs.

• Mentorship and talent investment initiatives.

• Internal mobility opportunities.

• Internship opportunities.

• Engaging projects for prominent global clients.

• An inclusive and supportive work atmosphere.

• Regular team-building social events.