Junior Data Engineer, Azure Databricks

This is a fully remote position, open to applicants in India, +1 more country.

📋 Description

• Support and enhance existing SQL Server data platforms and ETL solutions.

• Design and sustain data pipelines utilizing Azure Databricks.

• Construct and optimize data transformations with PySpark and SQL in Databricks.

• Create ETL/ELT pipelines managed through Azure Data Factory.

• Integrate data from various sources into the data platform and analytical layers.

• Maintain data models and data warehouse architectures for analytics.

• Ensure the quality, scalability, and performance of large-scale data processing pipelines.

• Collaborate with BI teams to support Power BI and reporting systems.

• Engage in the implementation of modern cloud-based data architectures with guidance from senior team members.

⛳️ Requirements

• Over 2 years of experience in Data Engineering, Data Warehouse development, or a related area.

• Proficient in Azure Databricks.

• Experience in developing data pipelines using PySpark and Spark SQL.

• Knowledge of distributed data processing and big data principles.

• Strong SQL capabilities and experience with SQL Server relational databases.

• Experience in building data pipelines with Azure Data Factory.

• Familiarity with data processing and performance optimization techniques.

• Excellent analytical and problem-solving abilities.

• Strong communication and teamwork skills.

• A proactive team player.

• Willingness to learn and continually enhance technical skills.

• Adaptable and comfortable in dynamic work environments.

• Knowledge of Spark optimization techniques (such as partitioning, caching, and cluster tuning) is a plus.

• Familiarity with Power BI and Databricks Structured Streaming is a plus.

• Experience transitioning from traditional ETL processes to cloud architectures is a plus.

• Familiarity with Delta Lake and Lakehouse architectures is a plus.

🏝️ Benefits

• Health insurance.

• Language courses.

• Relocation program.

• Flexible remote work options through the ForeverRemote work culture.

• Opportunities for professional development.

• Certification programs.

• Mentorship and talent investment initiatives.

• Internal mobility opportunities.

• Internship opportunities.

• Chance to work on impactful projects for top global clients.

• Inclusive and supportive multicultural work environment.

• Regular team-building social events.

• Commitment to sustainable business practices focused on IT education, community empowerment, fair operating practices, environmental sustainability, and gender equality.