Junior Data Engineer, Azure Databricks

This is a fully remote position, open to applicants in India, +1 more country.

📋 Description

• Assist in the enhancement and maintenance of current SQL Server data platforms and ETL solutions.

• Create and oversee data pipelines utilizing Azure Databricks.

• Construct and refine data transformations using PySpark and SQL within Databricks.

• Develop ETL/ELT pipelines managed through Azure Data Factory.

• Incorporate data from various sources into the data platform and analytical layers.

• Sustain data models and data warehouse frameworks for analytics purposes.

• Ensure the quality, scalability, and efficiency of large-scale data processing pipelines.

• Work with BI teams to assist with Power BI and reporting platforms.

• Engage in the implementation of modern cloud-based data architectures under the mentorship of senior team members.

⛳️ Requirements

• A minimum of 2 years of experience in Data Engineering, Data Warehouse development, or a related discipline.

• Proficiency with Azure Databricks.

• Experience in developing data pipelines utilizing PySpark and Spark SQL.

• Understanding of distributed data processing and big data principles.

• Strong SQL skills and familiarity with SQL Server relational databases.

• Experience in constructing data pipelines using Azure Data Factory.

• Exposure to concepts related to data processing and performance optimization.

• Knowledge of Spark optimization techniques (such as partitioning, caching, and cluster tuning) is desirable.

• Familiarity with Power BI and Databricks Structured Streaming is a plus.

• Experience in transitioning from traditional ETL processes to cloud-based architectures is advantageous.

• Familiarity with Delta Lake and Lakehouse architectures is a bonus.

• Strong analytical and problem-solving capabilities.

• Effective communication and teamwork skills.

• A proactive team player.

• Willingness to learn and continuously enhance technical skills.

• Ability to adapt and thrive in dynamic environments.

🏝️ Benefits

• Comprehensive compensation and benefits package.

• Health insurance.

• Language courses.

• Relocation program.

• Remote work flexibility through the ForeverRemote work culture.

• Opportunities for professional development.

• Certification programs.

• Mentorship and talent investment initiatives.

• Internal mobility opportunities.

• Internship opportunities.

• Collaboration on impactful projects for leading global clients.

• An inclusive and supportive work environment.

• Regular team-building company social events.