Senior Data Engineer – AWS, LLMs

📋 Description

• Define and advance cloud-native data architecture on AWS;

• Design optimized data models for operational querying (APIs) and conversational agent consumption;

• Lead the migration of historical data from mainframe systems;

• Ensure consistency, integrity, and traceability;

• Work with large volumes and complex historical data;

• Build and maintain data ingestion and transformation pipelines;

• Integrate existing flows (PeopleSoft, mainframe, RDS);

• Optimize queries in PostgreSQL;

• Implement partitioning, indexing, and caching strategies (Redis);

• Reduce latency for frequent queries;

• Ensure compliance with LGPD and retention policies;

• Implement data access controls, masking/ anonymization when necessary, auditing, and lineage;

• Structure data for consumption by LLMs;

• Support the creation of semantic layers;

• Work with embeddings/semantic indexing (a plus);

• Implement data quality validations;

• Monitor pipelines and maintain consistency;

⛳️ Requirements

• Experience in Data Engineering within AWS;

• Construction and maintenance of data pipelines (ingestion and transformation);

• Data integration from legacy and corporate systems (mainframe, PeopleSoft, RDS);

• Experience in data modeling for various consumption types (APIs and analytics/AI);

• Strong expertise in PostgreSQL (query optimization, performance, indexing, and partitioning);

• Knowledge of caching (Redis);

• Experience with large data volumes and legacy system migrations;

• Knowledge of data governance practices (LGPD, auditing, lineage, access control, and quality);

• Structuring data for consumption by LLMs (semantic layers and embeddings – an advantage);

🏝️ Benefits

• Competitive salary;

• Opportunities for professional development;

• Flexible work environment;

• Health and wellness programs;

• Collaborative and innovative team culture;