This is a fully remote position, open to applicants in Brazil.

• Design and implement pipelines and processes to gather, transform, and analyze cost and usage data from the data platform;

• Create metrics, dashboards, and indicators to assess resource efficiency and data utilization;

• Assist in developing cost monitoring solutions, detecting anomalies, and identifying opportunities for optimization;

• Analyze usage data from platforms like Databricks and AWS to gain insights into resource utilization;

• Contribute to the development of tools and processes that simplify cost management for data domains;

• Collaborate with engineers and architects to pinpoint areas for improvement in platform usage and suggest optimizations;

• Support initiatives related to cost governance and efficiency of the data platform;

• Engage in creating solutions that promote democratized access to platform usage and cost data;

• Experience or a keen interest in data engineering (including data pipelines, processing, and data modeling);

• Proficiency in SQL for data analysis and manipulation;

• Familiarity with Python for scripting and developing data pipelines;

• Awareness of cloud data environments such as AWS, GCP, or Azure;

• Eagerness to learn how data platforms are utilized and how to enhance their usage;

• Basic understanding of concepts related to Data Lake, Data Warehouse, or Lakehouse;

• Experience or interest in working with Spark or distributed processing;

• Capability to analyze data to uncover patterns, issues, or improvement opportunities;

• Strong communication skills and the ability to collaborate with multidisciplinary teams;

• n/a

Data Engineer – Specialist

People also viewed