Senior Data Engineer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design, construct, and sustain scalable data pipelines and integration workflows utilizing AWS, Snowflake, SQL, and Python.

• Oversee data ingestion from APIs, databases, SFTP locations, and file-based sources.

• Develop and enhance data models, transformations, and data quality controls.

• Manage recurring client processes, including the monthly SAS-based healthcare claims workflow.

• Monitor and resolve issues within pipelines, batch processes, and production data environments.

• Implement security, governance, access control, and credential management best practices.

• Enhance pipeline resilience through effective monitoring, error handling, restartability, and recovery.

• Optimize data workloads for performance, scalability, and cost efficiency.

• Create reusable components and patterns for ingestion, transformation, validation, and monitoring.

• Prepare governed and trusted datasets for analytics, reporting, AI, and machine learning applications.

• Collaborate with clients and internal stakeholders to convert business requirements into technical solutions.

• Modernize legacy processes, minimize manual effort, and boost operational efficiency.

• Maintain technical documentation, data lineage, business rules, and operational runbooks.

⛳️ Requirements

• Minimum of 5 years of experience in data engineering, ETL/ELT, or data integration.

• Strong expertise with AWS data services, such as S3 and RDS.

• Practical experience with Snowflake, SQL, and data modeling.

• Proficient in Python or a comparable scripting language.

• Experience in building and supporting scalable batch data pipelines.

• Familiar with integrating APIs, databases, SFTP locations, and file-based sources.

• Strong grasp of data transformation, validation, quality, and governance principles.

• Experience in monitoring, troubleshooting, and enhancing production data processes.

• Capability to work independently and take ownership of client-facing deliverables.

• Excellent problem-solving, documentation, and communication abilities.

• Experience with SAS datasets and data-processing programs.

• Possess a valid Microsoft certification or commit to obtaining a relevant certification within three months of employment.

• Experience with Secure FTP/SFTP and Gzip-compressed files is a plus.

• Familiarity with processing large, partitioned datasets is a plus.

• Knowledge of server-based batch processing is a plus.

• Experience with medical benefit or pharmacy benefit claims data is a plus.

• Familiarity with Databricks, AI, machine learning, or generative AI applications is a plus.

• Understanding of CI/CD, monitoring, and automation practices is a plus.

• Experience with Power BI or similar analytics and visualization tools is a plus.

• Background in regulated or security-sensitive environments is a plus.

🏝️ Benefits

• Invest in employee learning and long-term career growth.

• Opportunity to pursue a relevant Microsoft certification within three months of joining.