Senior Data Engineer, Data Quality Focus

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Develop and execute data quality processes throughout the data engineering lifecycle.

• Document, test, validate, and approve the pipelines and solutions that are developed.

• Ensure the data provided to end users is of high quality, consistent, and reliable.

• Create solutions utilizing Python, SQL, and PySpark within a Databricks environment.

• Construct and sustain notebooks and data processing pipelines.

• Engage in requirements analysis alongside business stakeholders.

• Conduct unit testing and assist in the validation and approval processes for solutions.

• Investigate, design, and create innovative data engineering solutions.

• Consistently enhance the reliability, efficiency, and quality of processes and data.

• Share technical expertise and best practices regarding data engineering and data quality with the team.

• Assist in ensuring that data generates value for business sectors and aids in decision-making.

⛳️ Requirements

• At least 4 years of experience in Data Engineering.

• Proficient in Python and SQL for data transformation, analysis, and cleansing.

• Familiarity with PySpark.

• Experience working with Databricks, including the development of notebooks in Python and PySpark.

• Background in testing, validation, and approval procedures for data solutions.

• Experience in requirements analysis and the development of data engineering solutions.

• Understanding of DevOps tools and practices, including Git, Docker, Jira, Bitbucket, Bamboo, Nexus, and microservices.

• Dedication to delivering quality and adhering to development best practices.

🏝️ Benefits

• Meal and food vouchers (Swile)

• Flexible credit for home office allowance (Swile)

• Up to 100% coverage for Health and Dental plans

• Group life insurance

• Remote work

• Mental health program — online and in-person psychotherapy

• Support for certifications and courses

• Tuition assistance for postgraduate and MBA courses (ESALQ/USP)

• Partnerships with language schools

• Partnerships with gyms and wellness apps (Wellhub)

• Internal talks and discussion sessions

• Referral bonus

• Happy hours

• Gifts on special occasions