Mid-Level Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design, implement, and enhance ETL (Extract, Transform, Load) pipelines to ensure a smooth data flow between systems.

• Utilize tools like Apache Airflow to orchestrate and schedule data processing tasks effectively.

• Develop and manage functions within cloud environments (such as AWS Lambda and Google Cloud Functions) to automate workflows and manage data.

• Integrate cloud data services to enhance scalability and efficiency.

• Set up and oversee Grafana dashboards to monitor the performance of data pipelines and identify potential bottlenecks.

• Establish alerts and metrics to guarantee data integrity and accessibility.

• Document data processes and routines to ensure clarity and facilitate maintenance.

• Engage in Engineering Change Proposals (ECPs) to suggest enhancements and innovations to workflows and data architecture.

• Collaborate closely with data science teams, analysts, and developers to understand requirements and ensure solutions align with business needs.

• Take part in planning and review meetings to synchronize objectives and priorities.

• Implement and oversee data validation processes to ensure the quality and accuracy of information.

• Conduct testing and audits of data pipelines to identify and address issues.

⛳️ Requirements

• Expertise in SQL and programming languages such as Python or Java.

• Experience with ETL and data integration tools.

• Strong understanding of cloud data architectures (e.g., AWS, Google Cloud, and Azure).

• Extensive knowledge of Apache Airflow for managing workflows.

• Ability to configure and utilize monitoring tools like Grafana.

• Familiarity with cloud services (e.g., S3, BigQuery, and Redshift) and best practices associated with them.

• Understanding of cost management and resource optimization within cloud environments.

🏝️ Benefits

• This position is also open to applicants with disabilities.