Azure Databricks Data Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Design and execute comprehensive data pipelines utilizing Azure-native services and Databricks.

• This position emphasizes the creation of scalable, high-performance data platforms, establishing robust data frameworks, and collaborating with stakeholders to convert business needs into dependable data solutions.

• Apply data governance principles through the use of Unity Catalog.

• Develop and implement data ingestion and integration patterns within a medallion architecture.

• Employ best practices in the field of data engineering.

• Generate PySpark notebooks.

• Formulate procedures for advanced data transformation.

• Address performance tuning considerations within Databricks and Azure SQL databases.

⛳️ Requirements

• Practical experience with Azure Databricks.

• Proficient in data ingestion, transformation, and orchestration frameworks.

• Practical experience in constructing insurance-specific data models.

• Demonstrated capability to enhance data pipelines for optimal performance, scalability, and cost-effectiveness.

• Strong grasp of distributed data processing concepts.

• Hands-on experience with PySpark for extensive data processing.

• Experience in defining and implementing best practices for Databricks workloads and Spark-based pipelines.

• Excellent analytical and problem-solving abilities.

• Outstanding communication skills to effectively engage with cross-functional stakeholders.

• Prior experience in insurance P&C cloud-based data and analytics projects.

🏝️ Benefits

• Diversity Inclusion: Exavalu also promotes flexibility, adapting to the needs of employees, customers, and the business. It might be part-time work, working outside normal 9-5 business hours, or working remotely.

• We also offer a welcome back program to assist individuals in reintegrating into the workforce after an extended absence due to health or family issues.

• At Exavalu, we are dedicated to cultivating a diverse and inclusive workforce. We encourage applications for employment from all qualified candidates, irrespective of race, color, gender, national or ethnic origin, age, disability, religion, sexual orientation, gender identity, or any other status protected by applicable law. We foster a culture that values all individuals and encourages diverse perspectives, where you can make a significant impact and advance your career.