Staff Data Engineer, Databricks Specialist

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Take ownership and lead the technical architecture for intricate, cross-team data projects that cover ingestion, transformation, storage, and serving layers.

• Design, construct, and sustain scalable, high-performance data pipelines and distributed data platforms within a cloud-native setting.

• Architect and oversee enterprise Master Data Management, which includes golden records, entity resolution, data domains, reference and hierarchy management, and stewardship.

• Design and integrate agentic AI and LLM-driven workflows, such as autonomous agents, RAG pipelines, and AI copilots, into data platforms and pipelines.

• Develop and support the foundational data structures necessary for machine learning and AI, including feature stores, vector stores, embeddings, and ML/LLMOps pipelines.

• Create and deliver backend data services along with REST/GraphQL APIs.

• Contribute across the technology stack to make curated datasets accessible to applications, analytics, and BI consumers.

• Set engineering standards regarding data quality, modeling, testing, observability, deployment, and the responsible use of AI-assisted development tools.

• Establish frameworks for data governance, lineage, cataloging, and quality.

• Lead technical design reviews and offer architectural guidance to various engineering and data teams.

• Collaborate with product, analytics, and engineering leadership to convert business strategy into scalable data roadmaps.

• Identify and address systemic performance, reliability, and scalability challenges across the data stack.

• Mentor and coach senior and mid-level engineers.

• Promote the adoption of modern frameworks, tools, and engineering practices.

• Stay informed about emerging technologies and industry trends in agentic AI, master data management, and modern data platforms.

⛳️ Requirements

• Bachelor's degree in Computer Science, Engineering, MIS, or a related field is preferred.

• 12+ years of experience in data engineering or software engineering.

• Proven experience in architecting data platforms and pipelines at scale.

• Expert-level SQL skills.

• Strong proficiency in Python; knowledge of Scala or Java is a plus.

• Extensive experience with cloud data platforms such as Databricks, Snowflake, Synapse, BigQuery, or Redshift.

• Comprehensive understanding of distributed systems, data modeling, and ETL/ELT architecture.

• Practical experience designing and implementing Master Data Management solutions, including entity resolution, match/merge, golden records, and reference/hierarchy management.

• Hands-on experience in building or integrating agentic AI systems, LLM-powered applications, RAG pipelines, or AI agent orchestration frameworks.

• Experience in creating backend data services and REST/GraphQL APIs.

• Strong background in relational and NoSQL data stores, in addition to data lake/lakehouse formats such as Delta, Iceberg, or Parquet.

• Profound understanding of CI/CD pipelines, infrastructure as code, and DevOps/DataOps practices.

• Demonstrated ability to lead large-scale technical initiatives across multiple teams.

• Proven track record of mentoring engineers and influencing technical direction without direct reporting authority.

• Nice-to-have experience with data governance, lineage, cataloging, multi-agent systems, event-driven architectures, streaming/real-time processing, ML/AI data layers, Docker, Kubernetes, Airflow, Dagster, dbt, and knowledge in commercial real estate, fintech, or operations/transaction systems.

🏝️ Benefits

• Work from anywhere — we hire and collaborate with professionals globally.

• Flexible working conditions and a remote work model.

• Engage in impactful projects across various industries.

• Continuous learning and development opportunities.

• Gain hands-on experience.

• Collaborate with a diverse and cross-cultural team.