Senior Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Lead the establishment and execution of data architecture.

• Ensure the quality, security, and governance of data.

• Enhance the performance and scalability of solutions.

• Construct and sustain scalable, effective data infrastructures.

• Integrate various data sources.

• Automate workflows for information ingestion and transformation.

• Create scalable data pipelines utilizing AWS (S3, EMR, Redshift).

• Engage in large-scale data processing utilizing dbt, DuckDB, and Spark.

• Participate in the development of dynamic DAGs with Airflow and YAML.

• Discuss requirements with stakeholders and suggest technical solutions.

• Monitor and ensure the integrity of pipelines and processed data.

⛳️ Requirements

• Extensive experience with AWS (S3, EMR, Redshift).

• Advanced skills in object-oriented Python and SQL.

• Familiarity with dbt, DuckDB, and Airflow.

• Experience in data architecture and understanding pipeline interdependencies.

• Experience in pipeline observability and monitoring is a plus.

• Proactive attitude with the ability to communicate effectively with requesting teams.

• Collaborative mindset along with technical leadership capabilities.

• Ability to comprehend cause-and-effect relationships within data flows.

• Previous experience as a software engineer is advantageous.

• Experience in high-volume environments and adherence to specific business rules is a plus.

• Knowledge of data quality tools such as Soda or Great Expectations is beneficial.

🏝️ Benefits

• Contractor arrangement.

• Remote work (as specified in the job header).