Principal Databricks Architect

This is a fully remote position, open to applicants in United States.

📋 Description

• Create, design, and deploy scalable architecture solutions on Databricks utilizing Delta Lake, Unity Catalog, MLflow, and related platform features for enterprise clients.

• Work closely with cloud solution architects, data engineers, data scientists, business analysts, and customer stakeholders to deliver intricate end-to-end enterprise data solutions.

• Collaborate with pre-sales teams to draft Databricks statements of work, which encompass solution scope, estimates, assumptions, timelines, and implementation strategies.

• Provide guidance to executive and technical stakeholders regarding the business advantages of Databricks and AI/ML, translating capabilities into tangible customer outcomes.

• Lead billable consulting efforts focused on Databricks architecture, cloud-native patterns, platform modernization, data governance, data quality, and cloud engineering.

• Advocate for AI/ML through automation, optimization, and machine learning frameworks to expedite implementation, lower costs, and enhance customer impact.

• Evaluate customer data ecosystems, cloud data preparedness, IT priorities, and business success metrics to design implementation architectures and structured learning plans.

• Act as an internal and external thought leader by influencing Databricks and AI/ML strategies, mentoring colleagues, contributing to reusable delivery assets, and fostering CDW’s reputation as a trusted advisor.

⛳️ Requirements

• Bachelor’s degree with over 10 years of consulting experience, including practical experience in designing and implementing Databricks platform solutions, OR 14+ years of relevant IT, data platform, or cloud architecture experience.

• Extensive knowledge of Databricks, Delta Lake, Unity Catalog, MLflow, data pipelines, ETL/ELT frameworks, batch and streaming solutions, PySpark, Scala, SQL, and Python scripting.

• Proficiency in data governance, data cataloging, master data management, data quality, RBAC, security, compliance, and data lineage.

• Strong experience in cloud data architecture across Azure, AWS, or Google Cloud.

• Familiarity with data platform modernization, cloud-native design patterns, advanced analytics, and enabling AI/ML solutions.

• Capability to enhance Databricks performance and cost-effectiveness through cluster configuration, job tuning, workload design, automation, and adherence to platform best practices.

• Demonstrated ability to lead architectural decisions, assume end-to-end technical responsibility, mentor team members, and guide platform-agnostic projects in agile environments.

• Exceptional communication, stakeholder management, problem-solving, adaptability, and executive presentation skills.

• Preferred certifications include Databricks Certified Data Engineer Professional and Databricks Certified AI/ML Engineer Associate or Professional.

• Openness to travel to other CDW locations or client sites.

• A valid U.S. driver’s license is required.

🏝️ Benefits

• Annual bonus target of 10% subject to terms and conditions of the plan.

• Comprehensive benefits package (overview linked).

• Flexible remote work arrangement.

• Opportunities for professional growth and learning.

• A collaborative and supportive work environment.

• Commitment to equal opportunity employment.