Senior Data Engineer – Databricks AI/BI, Genie

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design and execute analytics solutions utilizing Databricks AI/BI.

• Set up and enhance a Genie Space to facilitate natural-language data inquiries.

• Partner with business teams to establish metrics, prioritize questions, and identify use cases.

• Prepare, transform, and model the data utilized by the solutions.

• Develop and refine queries using Databricks SQL.

• Configure instructions, context, descriptions, relationships, and sample queries for Genie.

• Verify the accuracy and consistency of the responses generated by the conversational solution.

• Create dashboards and visualizations within the Databricks AI/BI environment.

• Ensure data quality, security, governance, and access control measures are in place.

• Identify and address issues related to data, queries, and Genie responses.

• Enhance the performance of queries and components utilized by the solution.

• Document the architecture, data sources, metrics, business rules, configurations, and procedures.

• Facilitate knowledge transfer to the involved teams.

• Collaborate effectively with Data, Analytics, Technology, and business teams.

⛳️ Requirements

• Extensive experience working at a senior level with Databricks.

• Practical experience with Databricks SQL, Delta Lake, and Lakehouse architecture.

• Familiarity with Unity Catalog, governance, and access control.

• Proficient in SQL for developing and optimizing queries.

• Experience with Python and/or PySpark for data preparation and transformation.

• Knowledge of data modeling for analytics solutions.

• Experience in creating pipelines and ETL/ELT processes.

• Ability to analyze issues, identify root causes, and propose effective solutions.

• Experience collaborating with business teams to grasp metrics and requirements.

• Excellent communication skills, self-sufficiency, and the capability to work on projects with defined deliverables.

• Databricks certifications in Data Engineering, Analytics, or AI (a plus).

• Background in consulting projects and direct client interaction (a plus).

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Opportunities for professional development and growth.

• Flexible working hours and remote work options.

• Comprehensive health and wellness benefits.

• Engaging work environment with a focus on innovation.