Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Engage in a project aimed at transitioning IBM DataStage processes to Python and PySpark within an AWS framework.

• Evaluate current solutions and comprehend business rules.

• Create scalable, efficient, and maintainable data pipelines.

• Contribute to the modernization of the data platform and influence technical and architectural decisions.

• Leverage Artificial Intelligence tools to aid in analysis, development, testing, and documentation, while critically assessing deliverables.

• Establish monitoring, failure handling, and reprocessing protocols.

• Conduct functional validation and quality assurance for migrated processes.

• Take part in code reviews and offer technical support to development teams.

• Share expertise and provide mentorship and professional growth opportunities for team members.

⛳️ Requirements

• Proficiency with AWS, particularly with Amazon S3, AWS Glue, Amazon EMR, or similar services.

• Extensive knowledge of Python and PySpark for constructing, maintaining, and optimizing data pipelines.

• Strong expertise in SQL, including the development, optimization, and tuning of complex queries.

• Experience in Data Architecture, data modeling, integration, ETL/ELT, governance, data quality, and information security.

• Background in migrating and modernizing data processes.

• Familiarity with code version control using Git and software engineering best practices.

• Experience in automated testing, technical documentation, and code organization.

• Knowledge of Artificial Intelligence applications in software development, including prompt creation, critical evaluation of outcomes, and validation of proposed solutions.

• Capability to implement monitoring, failure handling, and reprocessing protocols.

• Experience in functional validation and quality assurance for migrated processes.

• Involvement in code reviews and provision of technical support to development teams.

• Preferred: familiarity with IBM DataStage, PL/SQL databases, AWS databases, legacy-to-cloud migration, and large-scale enterprise environments.

🏝️ Benefits

• Multi-benefit card – choose how and where to utilize it.

• Scholarships available for undergraduate, graduate, MBA, and language courses.

• Incentive programs for certifications.

• Flexible working hours.

• Competitive salary packages.

• Annual performance assessments with a structured career development plan.

• Opportunities for international career advancement.

• Access to Wellhub and TotalPass.

• Private pension scheme.

• Childcare support.

• Medical coverage.

• Dental coverage.

• Life insurance.