
Data Engineer
Posted 6 days ago

Posted 6 days ago
This is a fully remote position, open to applicants in Brazil.
• Design, maintain, and enhance data processing systems and ETL pipelines utilizing tools such as Azure Data Factory and Azure Databricks.
• Partner with the Data Architect to establish data models and architectures that facilitate scalable and dependable data flows.
• Guarantee data quality and integrity through unit testing with Pytest and through component and integration testing with Behave.
• Collaborate with cross-functional teams to assimilate data from diverse sources by employing PySpark processes executed in Databricks and orchestrated by Azure Data Factory.
• Oversee infrastructure components within the Azure Cloud, incorporating role-based access control and Infrastructure as Code (IaC) methodologies.
• Participate in continuous integration and continuous deployment (CI/CD) processes utilizing Azure DevOps and related tools.
• Conduct data analysis and processing using SQL, Python, and Spark to meet business intelligence and reporting requirements.
• Over 5 years of experience in data engineering or a similar field.
• Expertise in SQL, Python, and Spark for data processing and analysis.
• Significant experience with Azure Data Factory and Azure Databricks in the development of ETL and data pipelines.
• Understanding of data modeling and architecture concepts.
• Familiarity with data quality tools and methodologies, including automated testing with PyTest and Behave.
• Basic knowledge of Azure Kubernetes Service (AKS) and event-driven architectures.
• Understanding of CI/CD methodologies and tools such as Azure DevOps, YAML, and shell scripting.
• Experience in infrastructure management within Azure, with a foundational knowledge of Terraform and role-based access control.
• Competitive salary and benefits package.
• Opportunity to join at the ground floor with a purpose-driven mindset.
• Development and career advancement paths to enhance your abilities.
• Attractive bonus potential from day one.
Aimpoint Digital
Power Digital Marketing
Get handpicked remote jobs straight to your inbox weekly.