Senior Data Engineer – Pipeline

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design and facilitate data flows, including ETL, ELT, and streaming data processes.

• Create and sustain data pipelines for the efficient and reliable transfer of information across systems.

• Conduct testing and implement tools to automate and enhance both real-time and batch data processing.

• Maintain data integrity and quality throughout the entire data lifecycle.

• Utilize Python for data manipulation, particularly with tools such as Pandas, NumPy, PySpark, Polars, DuckDB, SQLAlchemy, or similar technologies.

• Engage in Machine Learning projects utilizing Airflow, Spark, Flink, Kafka, Parquet, and other comparable tools.

⛳️ Requirements

• Bachelor's degree in Information Technology or a related discipline, or a degree accompanied by postgraduate studies in computing/IT.

• Hands-on experience with databases and proficiency in SQL.

• Familiarity with multidimensional databases.

• Understanding of ETL (Extract, Transform, Load) processes.

• Knowledge of data structures, data warehouses, and data marts.

• General understanding of Business Intelligence concepts.

• Strong analytical abilities.

• Expertise in Microsoft Power BI.

• Experience in designing and implementing data flows, including ETL, ELT, and data streaming.

• Proven experience in developing and maintaining data pipelines.

• Background in testing and deploying tools for real-time and batch data processing.

• Advanced proficiency in Python for data, particularly with Pandas, NumPy, PySpark, Polars, DuckDB, and SQLAlchemy, or similar frameworks.

• Advanced understanding of Machine Learning projects, particularly with Airflow, Spark, Flink, Kafka, and Parquet, among other tools.

🏝️ Benefits

• Meal allowance or meal voucher.

• Discounts for courses, universities, and language institutions.

• Stefanini Academy — an online platform providing free, contemporary courses with certification.

• Mentorship opportunities.

• Benefits club for consultations and exams.

• Health insurance coverage.

• Dental insurance options.

• Perks and discounts at partner establishments.

• Access to a travel club.

• Pet care benefits.