Senior Databricks/Python Developer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Create and implement scalable data pipelines utilizing Databricks (PySpark) for the processing and analysis of extensive datasets.

• Work closely with engineering and data science teams to grasp requirements and provide effective solutions.

• Execute intricate data transformations, cleansing, and aggregations using PySpark while enhancing code performance.

• Utilize Python ElementTree for effective XML data handling and integration of diverse data types.

• Craft Python scripts with Pandas for structured data manipulation and analytical purposes.

• Ensure the quality and integrity of data by adopting testing and monitoring methodologies.

• Play a key role in establishing and applying best practices in data engineering and data architecture.

• Maintain detailed technical documentation for processes and solutions that have been implemented.

⛳️ Requirements

• Proven experience in developing data solutions using Python.

• Demonstrated experience in constructing and managing data pipelines with Databricks (PySpark).

• Experience in processing, transforming, and analyzing substantial volumes of data.

• Proficiency in manipulating and processing data with Pandas.

• Familiarity with reading, processing, and integrating XML data using Python ElementTree.

• Experience with cloud platforms, particularly Microsoft Azure.

• Understanding of data quality, integrity, and monitoring standards.

• Experience in defining and applying best practices in data engineering and architecture.

• Experience collaborating with Data Engineering, Analytics, and Data Science teams.

• Knowledge of performance optimization for data pipelines and distributed processing.

• Familiarity with technical documentation related to data processes, pipelines, and solutions.

🏝️ Benefits

• Meal and/or food allowance.

• Health and dental insurance.

• Pet health insurance.

• Home office allowance (for remote roles).

• Welcome kit provided during onboarding.

• Life insurance.

• Zenklub access, offering psychological and nutritional support.

• TotalPass membership.

• Certification and continuous training programs with partners (Adobe, Anthropic, AWS, Google, Microsoft, Salesforce).

• Partnerships and discounts for your professional growth.

• Access to UOL content.

• Maternity kit available.

• Extended maternity and paternity leave.

• Referral program.