Junior Data Engineer, Azure Databricks

This is a fully remote position, open to applicants in India, +1 more country.

📋 Description

• Support and enhance current SQL Server data platforms along with ETL solutions.

• Design and maintain data pipelines utilizing Azure Databricks.

• Construct and refine data transformations using PySpark and SQL within Databricks.

• Create ETL/ELT pipelines managed through Azure Data Factory.

• Consolidate data from various sources into the data platform and analytical layers.

• Oversee data models and data warehouse configurations for analytics purposes.

• Guarantee data quality, scalability, and the efficiency of extensive data processing pipelines.

• Work in partnership with BI teams to assist Power BI and reporting platforms.

• Engage in the implementation of modern cloud-based data architectures under the direction of senior team members.

⛳️ Requirements

• A minimum of 2 years of experience in Data Engineering, Data Warehouse development, or a similar field.

• Proficiency with Azure Databricks.

• Experience in developing data pipelines using PySpark and Spark SQL.

• Knowledge of distributed data processing and big data concepts.

• Strong SQL skills and familiarity with SQL Server relational databases.

• Experience in building data pipelines using Azure Data Factory.

• Understanding of data processing and performance optimization methodologies.

• Awareness of Spark optimization strategies, including partitioning, caching, and cluster tuning.

• Familiarity with Power BI and Databricks Structured Streaming.

• Experience in transitioning traditional ETL processes to cloud-based architectures.

• Knowledge of Delta Lake and Lakehouse architectures.

• Excellent analytical and problem-solving abilities.

• Good communication and collaboration skills.

• Proactive, adaptable, and eager to learn.

• Capacity to thrive in dynamic environments.

🏝️ Benefits

• Comprehensive compensation and benefits package.

• Health insurance.

• Language courses.

• Relocation program.

• Remote work flexibility.

• Opportunities for professional development.

• Certification programs.

• Mentorship and talent investment initiatives.

• Internal mobility opportunities.

• Internship opportunities.

• Engaging projects for leading global clients.

• An inclusive and supportive work environment.

• Regular team-building social events within the company.