Data Engineer – Specialist

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Define and execute Artificial Intelligence solutions for data modernization and legacy systems.

• Create methods for analyzing, interpreting, and extracting technical information from legacy artifacts.

• Develop Generative AI-based solutions to expedite documentation, transformation, and migration processes.

• Construct intermediate metadata models to illustrate flows, business rules, dependencies, entities, and transformations.

• Create accelerators and reusable components that align with the enterprise data architecture.

• Assist in defining templates, technical standards, and declarative structures for modern pipelines.

• Design and enhance pipelines utilizing Databricks, PySpark, Lakeflow Jobs, and Declarative Pipelines.

• Manage batch loads, incremental ingestions, CDC (Change Data Capture), and enterprise integrations.

• Promote the enhancement of data governance, traceability, and data quality throughout the migration process.

• Collaborate with architects, data engineers, and platform specialists to establish scalable and secure solutions.

⛳️ Requirements

• Proven experience with Artificial Intelligence in Data Engineering, Software Engineering, or Technical Automation.

• Practical knowledge of Generative AI and Large Language Models (LLMs).

• Experience with:

• AI agents

• RAG (Retrieval-Augmented Generation)

• Prompt engineering

• Function calling

• Tool use

• Automated workflows

• Familiarity with AI orchestration frameworks:

• LangChain

• LangGraph

• Semantic Kernel

• CrewAI

• Or equivalent frameworks

• Proficiency in Python.

• Experience with PySpark and Spark SQL.

• Development of data pipelines in distributed environments.

• Advanced knowledge of SQL.

• Understanding of XML, JSON, and YAML.

• Experience in data modeling and Data Warehousing.

• Familiarity with Databricks environments.

🏝️ Benefits

• Position available for candidates with disabilities (PcD).