
Data Engineer
Posted 1 day ago

Posted 1 day ago
This is a fully remote position, open to applicants in Brazil, +1 more country.
• Implement, maintain, and support production data pipeline workflows within the platform based on solution designs and business needs.
• Design and develop data flows tailored for specific use cases, along with machine learning pipelines and data tools for analytics and data science teams.
• Construct and optimize data processing workflows utilizing Apache Spark, PySpark, Python (Pandas), SQL, Git, and Apache Airflow.
• Work with technologies from the Hadoop ecosystem, including Hive, Impala, HDFS, and Sqoop.
• Analyze structured and semi-structured datasets to comprehend data characteristics and aid in pipeline design and implementation.
• Enhance the performance, scalability, and reliability of big-data pipelines, processing architectures, and datasets.
• Deploy and manage data processing workloads in environments based on Kubernetes.
• Collaborate with clients and cross-functional teams to grasp requirements and deliver effective technical solutions.
• Employ a business-focused approach when designing and implementing data workflows.
• Utilize Large Language Models (LLMs), agentic coding, LLM cost optimization, and AI agent development to expedite delivery and create data tools for both internal and external applications.
• Strong hands-on experience with Python, including Pandas.
• Experience with Apache Spark and PySpark for distributed data processing.
• Proficient SQL skills and experience in analyzing large datasets.
• Practical experience with Apache Airflow and workflow orchestration.
• Familiarity with Hadoop ecosystem technologies such as Hive, Impala, HDFS, and Sqoop.
• Experience with Git and the development of production data pipelines.
• Understanding of structured and semi-structured data formats.
• Experience in optimizing data pipelines and workloads within Kubernetes environments.
• Exposure to LLMs, agentic coding, AI agents, and LLM cost optimization.
• Strong problem-solving abilities and skills in customer collaboration.
• Competitive compensation and benefits.
• Meaningful opportunities for career development.
• A diverse and inclusive culture.
• An environment that fosters collaboration, continuous learning, and growth.
• A challenging and engaging work atmosphere.
• The chance to work with a talented global team.
• Opportunities to contribute to significant technical initiatives.
• The ability to help shape a high-performing DevOps culture.
Trax Technologies
Fiserv
Ontrac Solutions
General Dynamics Information Technology
Get handpicked remote jobs straight to your inbox weekly.