Lead Data Engineer, Brazil

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design, implement, and enhance a petabyte-scale AWS data platform.

• Construct and optimize scalable data pipelines utilizing JVM languages, Spark, Python, and cloud-native services.

• Ensure efficient, resilient, and production-ready data processing for analytics and downstream products.

• Mentor engineers, conduct code reviews, and advocate for engineering best practices.

• Participate in architectural and design discussions with architects, product owners, and engineering leadership.

• Plan and execute complex technical initiatives within agile value-stream teams.

• Guarantee operational readiness through adherence to coding standards, testing, monitoring, and release procedures.

• Assist with production deployments and uphold platform stability.

• Investigate and integrate GraphQL, vector databases, and AI/LLM-based methodologies.

• Collaborate with product, engineering, and operations teams to facilitate data-driven marketing solutions.

⛳️ Requirements

• Extensive software and data engineering experience in large-scale data platforms.

• Profound experience in building petabyte-scale systems using Java-based languages, including Scala and Apache Spark.

• Strong expertise within the AWS data ecosystem, encompassing Glue, S3, Athena, Managed Airflow, and Iceberg.

• Advanced comprehension of distributed systems and highly parallelized workloads.

• Robust skills in query optimization, data partitioning, and efficient storage strategies.

• Mastery of the AWS cloud ecosystem; knowledge of Azure/GCP is advantageous.

• Experience in influencing long-term architecture and platform design.

• Exceptional code review, mentorship, and technical leadership abilities.

• Practical experience in agile environments and collaboration across various engineering teams.

• Strong version control and multi-repo collaboration expertise with Git, GitHub, and Bitbucket.

• Bachelor’s degree in Computer Science, Engineering, or a related discipline, or equivalent experience.

• Proficient in English.

• Willingness to travel to São Carlos/SP when required.

• Relevant professional experience.

• Familiarity with DBT or contemporary transformation frameworks.

• Understanding of concurrent and parallel programming.

• Knowledge of Python, Angular, and TypeScript.

• Experience with vector databases such as pgvector and Redis vector fields.

• Practical experience in applying AI/LLM techniques to data platforms, data access, or software quality.

🏝️ Benefits

• Inclusive recruitment and professional development initiatives.

• Affinity groups that support underrepresented communities: ExperianPride, Ubuntu, Women in Experian, Aspire, and Connecting Generations.

• Option for remote work.

• Full-time regular employee status.