Senior Data Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Design, develop, and maintain scalable and robust data solutions in the cloud utilizing Apache Spark and Databricks.

• Collect and evaluate data requirements from business stakeholders, identifying opportunities for data-driven insights.

• Construct and enhance data pipelines for ingestion, processing, and integration using Spark and Databricks.

• Maintain data quality, integrity, and security throughout all phases of the data lifecycle.

• Collaborate with cross-functional teams to devise and implement data models, schemas, and storage solutions.

• Improve data processing and analytics performance by fine-tuning Spark jobs and utilizing Databricks features.

• Offer technical guidance and expertise to junior data engineers and developers.

• Keep abreast of emerging trends and technologies in cloud computing, big data, and data engineering.

• Contribute to the ongoing enhancement of data engineering processes, tools, and best practices.

⛳️ Requirements

• Bachelor's or Master's degree in computer science, engineering, or a related discipline.

• Over 10 years of experience as a Data Engineer, Software Engineer, or in a similar position, with a focus on developing cloud-based data solutions.

• Strong experience with cloud platforms such as Azure.

• Proficient in Apache Spark and Databricks for large-scale data processing and analytics.

• Experience in the design and implementation of data processing pipelines using Spark and Databricks.

• Solid knowledge of SQL and experience with both relational and NoSQL databases.

• Experience with data integration and ETL processes using tools like Apache Airflow or cloud-native orchestration services.

• Good understanding of data modeling and schema design principles.

• Knowledge of data governance and compliance frameworks.

• Excellent problem-solving and troubleshooting abilities.

• Strong communication and collaboration skills for effective work within a cross-functional team.

• Relevant certifications in cloud platforms, Spark, or Databricks are considered a plus.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Opportunities for professional development and continuous learning.

• Flexible work hours and remote work options.

• Comprehensive health and wellness benefits.

• Engaging company culture with team-building activities.