Senior Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Develop, implement, and enhance a petabyte-scale data platform on AWS.

• Create and refine scalable data pipelines utilizing JVM languages, Spark, Python, and cloud-native services.

• Guarantee efficient, resilient, and production-ready data processing that supports analytics and downstream products.

• Serve as a principal-level technical leader, mentoring engineers and overseeing high-quality code reviews.

• Advocate for engineering best practices and elevate engineering maturity throughout the organization.

• Participate in architectural and design discussions with architects, product owners, and engineering leadership.

• Plan and execute complex technical projects within agile value-stream teams.

• Ensure operational readiness through adherence to coding standards, testing practices, monitoring strategies, and release protocols.

• Assist with production deployments and uphold platform stability.

• Investigate and integrate technologies such as GraphQL, vector databases, and AI/LLM-based techniques.

⛳️ Requirements

• Extensive software and data engineering experience in large-scale data platforms.

• Profound experience in constructing petabyte-scale systems using Java-based languages, including Scala and Apache Spark.

• Strong knowledge of the AWS data ecosystem, encompassing Glue, S3, Athena, Managed Airflow, and Iceberg.

• Advanced comprehension of distributed systems and highly parallelized workloads.

• Exceptional skills in query optimization, data partitioning, and efficient storage methodologies.

• Mastery of the AWS cloud ecosystem; familiarity with Azure/GCP is advantageous.

• Experience in influencing long-term architecture and platform design.

• Outstanding code review, mentorship, and technical leadership abilities.

• Practical experience in agile environments and collaboration across multiple engineering teams.

• Strong version control and multi-repository collaboration skills using Git, GitHub, and Bitbucket.

• Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent experience.

• Relevant work experience.

• Proficient in English.

• Willingness to travel to São Carlos/SP when necessary.

• Familiarity with DBT or modern transformation frameworks.

• Knowledge of concurrent and parallel programming.

• Experience with Python, Angular, and TypeScript.

• Familiarity with vector databases like pgvector and Redis vector fields.

• Practical experience applying AI/LLM techniques to data platforms, data access, or software quality.

🏝️ Benefits

• Opportunities for remote work.

• Initiatives for inclusive recruitment and professional development.

• Affinity groups that support underrepresented communities: ExperianPride, Ubuntu, Women in Experian, Aspire, and Connecting Generations.