Lead Data Platform Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design, implement, and enhance a petabyte-scale data platform on AWS.

• Construct and optimize scalable data pipelines utilizing JVM languages, Spark, Python, and cloud-native services.

• Serve as a principal-level technical leader by mentoring engineers, performing code reviews, and advocating for best practices.

• Participate in architectural and design discussions with architects, product owners, and engineering leadership.

• Plan and execute complex technical projects within agile value-stream teams.

• Ensure operational readiness through adherence to coding standards, testing, monitoring, and release protocols.

• Support production deployments and uphold platform stability.

• Investigate and implement GraphQL integrations, vector databases, and AI/LLM-based techniques.

⛳️ Requirements

• Extensive software and data engineering expertise in large-scale data platforms.

• Profound experience in developing petabyte-scale systems utilizing Java-based languages such as Scala and Apache Spark.

• Strong proficiency in AWS data services, including Glue, S3, Athena, Managed Airflow, and Iceberg.

• Advanced comprehension of distributed systems and highly parallelized workloads.

• Solid skills in query optimization, data partitioning, and effective storage patterns.

• AWS cloud expertise; experience with Azure/GCP is a plus.

• Proven ability to influence long-term architecture and platform design.

• Exceptional code review, mentorship, and technical leadership skills.

• Hands-on experience in agile environments and collaboration across various engineering teams.

• Strong version control and multi-repository collaboration skills using Git, GitHub, and Bitbucket.

• Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent experience.

• Proficient in English.

• Willingness to travel to São Carlos/SP as necessary.

• Nice to have: Knowledge of DBT or modern transformation frameworks.

• Nice to have: Understanding of concurrent and parallel programming.

• Nice to have: Familiarity with Python, Angular, and TypeScript.

• Nice to have: Experience with vector databases, including pgvector and Redis vector fields.

• Nice to have: Practical experience applying AI/LLM techniques to data platforms, data access, or software quality.

🏝️ Benefits

• Remote work opportunities.

• Initiatives for inclusive recruitment and professional development.

• Affinity groups supporting underrepresented communities: ExperianPride, Ubuntu, Women in Experian, Aspire, and Connecting Generations.