Senior Databricks Architect

This is a fully remote position, open to applicants in United States.

📋 Description

• Design and implement comprehensive data solutions utilizing the Databricks Lakehouse Platform.

• Create and develop scalable data pipelines employing PySpark, Spark, and SQL.

• Lead the implementation of cloud data platforms on AWS, Azure, or GCP.

• Propel the modernization of traditional data systems into cloud-native architectures.

• Apply the Medallion Architecture, encompassing Bronze, Silver, and Gold layers.

• Ensure adherence to data governance, security, and compliance, particularly within Financial Services.

• Collaborate effectively with business stakeholders, data scientists, and engineering teams.

• Enhance performance, cost-efficiency, and reliability of data pipelines.

• Mentor and guide data engineering teams, providing architectural leadership.

⛳️ Requirements

• Over 12 years of experience in Data Engineering or Data Architecture.

• Profound expertise in Databricks, PySpark, and the Spark ecosystem.

• Practical experience with cloud platforms such as Azure, AWS, or GCP.

• Familiarity with data orchestration tools like Airflow, ADF, etc.

• Strong foundation in data modeling, ETL/ELT processes, and distributed systems.

• Experience with Delta Lake and Unity Catalog is advantageous.

• In-depth knowledge of the Financial Services domain, including Banking, Insurance, and Capital Markets.

• Exceptional problem-solving abilities and stakeholder management skills.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health and wellness benefits.

• Opportunities for professional development and continuous learning.

• Flexible work arrangements and a supportive work environment.