Data Engineer – AI

This is a fully remote position, open to applicants in Brazil, +5 more countries.

📋 Description

• Develop, construct, and enhance scalable ETL/ELT pipelines utilizing Apache Spark and Delta Lake across AWS, Azure, and GCP.

• Create scalable Lakehouse architectures capable of processing structured and unstructured data at an enterprise level.

• Streamline data ingestion, storage, and feature engineering processes for analytics and real-time AI applications.

• Optimize Spark jobs for minimal latency, maximum throughput, and cost efficiency in the cloud.

• Collaborate with ML engineers and data scientists to put AI models into operation in Databricks using MLflow.

• Develop CI/CD pipelines and Infrastructure-as-Code with Terraform and Databricks CLI.

• Uphold enterprise data security, access controls, and compliance standards, including GDPR, HIPAA, and SOC 2.

• Work in partnership with client technology leaders, product managers, and engineering teams to convert AI data needs into business results.

• Create and maintain technical documentation.

• Oversee solution transitions to client engineering teams.

⛳️ Requirements

• Over 3 years of experience in cloud-native data engineering.

• At least 2 years of hands-on experience with Databricks, Apache Spark, Delta Lake, and MLflow.

• Strong expertise in Python, SQL, Spark frameworks, CI/CD, and Infrastructure-as-Code (Terraform or Databricks CLI).

• Demonstrated success in fast-paced Agile/DevOps consulting environments.

• Experience in delivering cloud-agnostic architectures.

• Excellent communication skills with clients.

• Active certification as a Databricks Certified Data Engineer (Associate or Professional) or Databricks ML Professional is highly preferred.

🏝️ Benefits

• Remote-First Working Environment – Collaborate from anywhere while staying connected with our global team.

• Collaborative, Innovative Culture – Work alongside top talent in a fast-paced, supportive environment that values curiosity and initiative.

• Equal opportunity workplace.

• Reasonable accommodation for individuals with disabilities.