Senior AI Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design and Develop Data Pipelines: Architect, implement, and maintain scalable and high-performance ETL/ELT pipelines for processing large volumes of structured and unstructured data.

• Data Integration and Processing: Build event-driven architectures to enable real-time data processing and seamless integration between systems.

• Cloud Infrastructure Management: Leverage AWS cloud services (e.g., S3, Redshift, Glue, Lambda, Kinesis, Pentaho) to design and deploy a robust data infrastructure.

• Code Development and Optimization: Write clean, efficient, and maintainable code in Python to support data ingestion, transformation, and orchestration.

• Collaboration in Agile Environments: Work closely with cross-functional teams, including data scientists, analysts, and software engineers, following agile methodologies to deliver iterative solutions.

• Data Quality and Governance: Implement best practices for data quality, monitoring, and compliance to ensure data integrity, consistency, and security.

• Performance Optimization: Optimize data pipelines and queries in cloud environments for performance, scalability, and cost efficiency.

• Mentorship and Leadership: Provide technical guidance to junior team members, fostering a culture of continuous learning and improvement.

• Documentation and Knowledge Sharing: Document technical designs, processes, and workflows to ensure maintenance and knowledge transfer.

⛳️ Requirements

• Professional experience as a Data Engineer or in a similar role, with a strong emphasis on developing data pipelines and cloud-based data solutions.

• Advanced proficiency in Python for data processing, scripting, and automation.

• Hands-on experience with Databricks for big data processing, including Spark, Delta Lake, and Databricks workflows.

• In-depth knowledge of AWS services for building and managing data infrastructure.

• Expertise in designing and implementing ETL/ELT workflows using tools such as Apache Airflow, AWS Glue, or similar.

• Experience building real-time data pipelines using event-driven frameworks.

• Strong experience in agile environments, utilizing Scrum or Kanban methodologies.

• Proficiency in relational and non-relational databases, with expertise in SQL optimization.

• Solid understanding of data modeling techniques for analysis and reporting.

• Familiarity with CI/CD pipelines, version control, and infrastructure as code.

• Exceptional analytical skills focused on delivering scalable and efficient solutions.

• Excellent verbal and written communication skills, with the ability to collaborate effectively with both technical and non-technical stakeholders.

🏝️ Benefits

• Health and dental insurance;

• Meal and food vouchers;

• Childcare assistance;

• Extended parental leave;

• Partnership with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass;

• Profit Sharing (PLR);

• Life insurance;

• Continuous learning platform (CI&T University);

• Discount club;

• Free online platform dedicated to promoting physical, mental health, and well-being;

• Courses on pregnancy and responsible parenting;

• Partnerships with online course platforms;

• Language learning platform;

• And many more.