Junior Data Engineer, Azure Databricks

This is a fully remote position, open to applicants in India, +1 more country.

📋 Description

• Assist in enhancing and maintaining current SQL Server data platforms and ETL solutions.

• Create and manage data pipelines utilizing Azure Databricks.

• Design and refine data transformations using PySpark and SQL within Databricks.

• Develop ETL/ELT pipelines managed through Azure Data Factory.

• Consolidate data from various sources into the data platform and analytical layers.

• Sustain data models and data warehouse frameworks for analytics.

• Guarantee data quality, scalability, and performance for large-scale data processing pipelines.

• Work collaboratively with BI teams to assist with Power BI and reporting platforms.

• Engage in the implementation of modern cloud-based data architectures under the mentorship of senior team members.

⛳️ Requirements

• A minimum of 2 years of experience in Data Engineering, Data Warehouse development, or a related area.

• Proficiency with Azure Databricks.

• Experience in developing data pipelines with PySpark and Spark SQL.

• Understanding of distributed data processing and big data concepts.

• Strong SQL skills and experience with SQL Server relational databases.

• Proven experience in building data pipelines using Azure Data Factory.

• Exposure to data processing and performance optimization techniques.

• Knowledge of Spark optimization strategies, such as partitioning, caching, and cluster tuning (preferred).

• Familiarity with Power BI and Databricks Structured Streaming (preferred).

• Experience in transitioning traditional ETL processes to cloud architectures (preferred).

• Familiarity with Delta Lake and Lakehouse architectures (preferred).

• Excellent analytical and problem-solving abilities.

• Effective communication and collaboration skills.

• Team-oriented individual with a proactive approach.

• Willingness to learn and continually enhance technical skills.

• Adaptable and comfortable working in fast-paced environments.

🏝️ Benefits

• Competitive salary and benefits package.

• Health insurance coverage.

• Language learning courses.

• Relocation assistance program.

• Flexible remote work options through the ForeverRemote work culture.

• Opportunities for professional development.

• Access to certification programs.

• Mentorship and investment in talent development.

• Internal mobility opportunities.

• Internship programs available.

• Collaboration on meaningful projects for leading global clients.

• An inclusive and supportive multicultural work environment.

• Regular team-building and company social events.