Data Engineer

Posted 1 day ago

This is a fully remote position, open to applicants in Brazil, +1 more country.

📋 Description

• Implement, maintain, and support production data pipeline workflows within the platform based on solution designs and business needs.

• Design and develop data flows tailored for specific use cases, along with machine learning pipelines and data tools for analytics and data science teams.

• Construct and optimize data processing workflows utilizing Apache Spark, PySpark, Python (Pandas), SQL, Git, and Apache Airflow.

• Work with technologies from the Hadoop ecosystem, including Hive, Impala, HDFS, and Sqoop.

• Analyze structured and semi-structured datasets to comprehend data characteristics and aid in pipeline design and implementation.

• Enhance the performance, scalability, and reliability of big-data pipelines, processing architectures, and datasets.

• Deploy and manage data processing workloads in environments based on Kubernetes.

• Collaborate with clients and cross-functional teams to grasp requirements and deliver effective technical solutions.

• Employ a business-focused approach when designing and implementing data workflows.

• Utilize Large Language Models (LLMs), agentic coding, LLM cost optimization, and AI agent development to expedite delivery and create data tools for both internal and external applications.


⛳️ Requirements

• Strong hands-on experience with Python, including Pandas.

• Experience with Apache Spark and PySpark for distributed data processing.

• Proficient SQL skills and experience in analyzing large datasets.

• Practical experience with Apache Airflow and workflow orchestration.

• Familiarity with Hadoop ecosystem technologies such as Hive, Impala, HDFS, and Sqoop.

• Experience with Git and the development of production data pipelines.

• Understanding of structured and semi-structured data formats.

• Experience in optimizing data pipelines and workloads within Kubernetes environments.

• Exposure to LLMs, agentic coding, AI agents, and LLM cost optimization.

• Strong problem-solving abilities and skills in customer collaboration.


🏝️ Benefits

• Competitive compensation and benefits.

• Meaningful opportunities for career development.

• A diverse and inclusive culture.

• An environment that fosters collaboration, continuous learning, and growth.

• A challenging and engaging work atmosphere.

• The chance to work with a talented global team.

• Opportunities to contribute to significant technical initiatives.

• The ability to help shape a high-performing DevOps culture.

People also viewed

Trax Technologies8 hours ago

Data Architect

US flagUnited States OnlyFull-timeData Engineer
ApplyView job
Fiserv8 hours ago

Senior Snowflake Data Platform and Artificial Intelligence Architect

US flagUnited States OnlyFull-timeData Engineer
ApplyView job
Ontrac Solutions8 hours ago

Staff Data Engineer

US flagUnited States OnlyFreelanceData Engineer
ApplyView job
General Dynamics Information Technology8 hours ago

ICAM Identity Data Management Engineer – Radiant Logic

US flagUnited States OnlyFull-timeData Engineer$170k – $230k/year
ApplyView job
General Dynamics Information Technology8 hours ago

Identity Data Management Engineer – Radiant Logic

US flagUnited States OnlyFull-timeData Engineer$170k – $230k/year
ApplyView job
General Dynamics Information Technology8 hours ago

Senior Data Engineer

US flagUnited States OnlyFull-timeData Engineer$102k – $138k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers