Principal Architect – Databricks

This is a fully remote position, open to applicants in United States.

📋 Description

• Oversee the technical aspects of Databricks initiatives, which include qualification, technical exploration, target architecture design, effort estimation, and risk evaluation.

• Create technical documentation for proposals and Statements of Work (SOWs).

• Develop and sustain technical relationships with customer architects and stakeholders.

• Evaluate migration complexity, implementation efforts, AI workload readiness, and related risks.

• Review architectural decisions throughout the project delivery phase.

• Ensure that delivered solutions are consistent with the originally scoped and proposed solutions.

• Identify technical risks and issues early during engagements.

• Maintain involvement throughout the delivery process to ensure accountability for architecture and results.

• Define the technical content for market-facing Databricks offerings.

• Establish the scope, success metrics, and evidence to validate successful completion.

• Develop repeatable delivery standards and frameworks.

• Collaborate directly with Databricks field teams and specialist architects on shared customer accounts.

• Foster productive relationships with Databricks technical teams.

• Work collaboratively to address technical challenges, feedback, and differing perspectives from vendor architects and specialists.

• Design Databricks lakehouse platforms, including account and workspace design, Unity Catalog and metastore architecture, compute strategy, open table format decisions, and cost architecture.

• Lead migration and consolidation efforts from Hadoop, Spark, legacy analytics platforms, data warehouses, appliances, and other cloud platforms.

• Conduct platform assessments, wave planning, workload and code conversion, reconciliation and parity validation, as well as cutover planning and execution.

• Build and manage production-ready data engineering solutions and pipelines.

• Support production machine learning and AI engineering initiatives.

⛳️ Requirements

• Extensive expertise in lakehouse architecture and platform design.

• Extensive expertise in migration and consolidation processes.

• Extensive expertise in data engineering.

• Adequate proficiency in machine learning and AI engineering, governance at the perimeter, and consumption economics to identify when to engage specialist support.

• Significant hands-on experience with Databricks acquired through a partnership with Databricks, direct involvement with Databricks, or managing a large Databricks Lakehouse environment internally.

• Proven track record of successfully delivering enterprise-scale migration and consolidation projects.

• Capability to detail at least one major migration project, including lessons learned and outcomes.

• Experience in operating and supporting production machine learning or AI solutions beyond proof-of-concept implementations.

• Demonstrated understanding of operational, governance, and maintenance considerations relevant to production AI workloads.

• Proven success at the Principal Architect level or equivalent.

• Acknowledged as the senior technical authority in customer interactions.

• Trusted to commit organizations to scopes, architectures, and technical methodologies.

• Experience in presales environments or delivery leadership roles, showcasing the ability to lead customer discussions, define scope and solution strategies, develop proposals and Statements of Work, and uphold technical commitments made during the sales process.

• Comfortable collaborating with and being challenged by vendor architects and technical specialists.

• Databricks certification is preferred.

• Capable of obtaining Databricks certification quickly if not currently certified.

• Willingness to learn, adapt, and engage in continuous professional development.

• Strong leadership, strategic thinking, problem-solving, analytical thinking, prioritization, customer-centric mindset, organizational, communication, and interpersonal skills.

🏝️ Benefits

• Opportunities for continuous professional growth and leadership development.

• Health, wellness, and financial benefits.

• Medical insurance.

• Vision insurance.

• Dental insurance.

• 401K plan.

• Flexible spending options.

• Access to world-class facilities and the technology necessary to succeed – whether in our offices or yours.

• Support and funding for Databricks certification.