Solution Architect – Databricks

This is a fully remote position, open to applicants in United States.

📋 Description

• Create comprehensive cloud-native data architectures utilizing AWS and Databricks.

• Design scalable batch and real-time data processing solutions leveraging Apache Kafka and streaming technologies.

• Oversee the design and execution of enterprise data engineering pipelines and data platforms.

• Establish architecture standards, best practices, and governance for data integration and analytics solutions.

• Collaborate with business stakeholders to convert business requirements into scalable technical solutions.

• Provide technical leadership and architectural direction to development teams throughout the software development lifecycle (SDLC).

• Enhance data pipelines for performance, scalability, security, and cost-effectiveness.

• Develop data models and integration strategies for both structured and semi-structured data.

• Ensure solutions comply with enterprise security, compliance, and reliability standards.

• Assess emerging technologies and propose innovative solutions to enhance business capabilities.

• Facilitate cloud migration and modernization efforts.

• Mentor engineers and lead architecture and design reviews.

⛳️ Requirements

• Over 8 years of experience in software, data engineering, or solution architecture.

• Significant experience in designing enterprise data platforms on AWS.

• Practical expertise with Databricks and Lakehouse architecture.

• Experience in implementing Apache Kafka for real-time event streaming.

• Strong foundation in Data Engineering, including ETL/ELT pipeline development.

• Proven experience in building scalable real-time and streaming data processing solutions.

• Comprehensive understanding of distributed systems and cloud architecture principles.

• Proficiency in SQL, Python, or Scala.

• Familiarity with CI/CD, Infrastructure as Code, and cloud-native development methodologies.

• Exceptional communication and stakeholder management abilities.

• Databricks Certified Data Engineer Professional.

• Preferred experience with AI/ML platforms or deploying machine learning solutions in production.

• Preferred familiarity with Delta Lake, Spark Structured Streaming, or similar technologies.

• Preferred AWS Certification (Solutions Architect or Data Analytics).

• Additional experience with Terraform, Kubernetes, or Docker is advantageous.

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Comprehensive health and wellness benefits.

• Opportunities for professional development and growth.

• Flexible work hours and remote work options.

• Collaborative and innovative work environment.