This is a fully remote position, open to applicants in United States.

• Take charge of engagements from initial discovery through to final delivery, ensuring complete ownership and accountability.

• Design, implement, and optimize extensive Spark/Databricks workloads, prioritizing scalability and cost-effectiveness.

• Utilize advanced Spark principles to diagnose and fine-tune distributed data tasks and SQL workloads.

• Create and conduct Proofs of Concept (PoCs) and workshops to showcase solution strategies and build customer trust.

• Serve as a technical champion for clients, proactively addressing potential challenges with innovative, data-driven solutions.

• Work in collaboration with product and engineering teams to enhance customer experiences and improve platform functionalities.

• Promote best practices in PySpark, Databricks, and Lakehouse architecture while guiding junior engineers.

• Experience: 5–10 years in data engineering or delivering analytics solutions.

• Databricks Mastery: Over 3 years of practical expertise, including job optimization, debugging, and workload management.

• Technical Depth: Profound knowledge of Spark Internals, PySpark, and performance tuning for Databricks SQL (DBSQL).

• Modern Lakehouse: Comprehensive understanding of Delta Lake, Structured Streaming, and cloud infrastructure.

• Autonomy: Demonstrated capability to identify and resolve intricate technical issues with minimal supervision.

• Communication: Outstanding ability to convey complex technical concepts in a simplified manner to business leaders.

• Certifications: Preferably, possess Databricks Certified Data Engineer Professional credentials.

• Advanced Skills: Familiarity with Delta Live Tables (DLT) and Unity Catalog (UC) is a plus.

• Consulting Background: Experience in large-scale data migrations or AI-enhanced analytics within enterprise settings is advantageous.

• Remote work

• Health insurance

• Professional development

Resident Solutions Architect – MLOps

People also viewed