Senior Data Developer

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design and sustain data pipelines that cater to the Raw, Trusted, Refined, and Analytics layers.

• Propel the advancement of Medallion architecture and Unity Catalog, enhancing data governance and reliability.

• Execute integrations with Databricks and Azure Data Lake Storage (ADLS).

• Ingest and standardize data from various sources including APIs, databases, and files.

• Set up observability sensors to monitor platform health, performance, and integrity.

• Analyze ingestion and processing failures through log analysis, metrics, and focused testing.

• Address production incidents and implement solutions that minimize recurrence.

• Establish data quality controls and oversee data refresh routines.

• Enhance pipelines and processing resources for improved performance and cost-effectiveness.

• Document technical solutions and architectural decisions.

• Disseminate knowledge through code reviews and technical sessions.

⛳️ Requirements

• Extensive experience in constructing data pipelines within production environments, including incident management and troubleshooting.

• Proficiency in Python and SQL for developing and maintaining ETL/ELT solutions.

• Familiarity with large-scale distributed data processing utilizing Databricks and Spark.

• Practical experience with at least one major cloud platform: Azure, AWS, or GCP.

• Understanding of Lakehouse architecture, encompassing Raw, Trusted, Refined, and Analytics layers, along with Medallion architecture.

• Knowledge of data quality methodologies, monitoring, and troubleshooting in data settings.

• Acquaintance with version control, CI/CD, and software engineering best practices in data applications.

• Proficient in English.

• Experience with Unity Catalog or alternative data governance solutions in Lakehouse architectures is a plus.

• Knowledge of cloud cost optimization is advantageous.

• Previous experience in supporting or managing production data platforms is a plus.

• Certifications in Databricks, Azure, AWS, or GCP are welcome but not mandatory.

🏝️ Benefits

• Health and dental insurance.

• Meal and food allowance.

• Childcare assistance.

• Extended paternity leave.

• Partnership with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass.

• Profit Sharing and Results Participation (PLR).

• Life insurance.

• Continuous learning platform (CI&T University).

• Discount club.

• Free online platform dedicated to physical, mental, and overall well-being.

• Pregnancy and responsible parenting course.

• Partnerships with online learning platforms.

• Language learning platform.

• Health and well-being team, inclusion specialists, and affinity groups.

• Support and accommodations for individuals with disabilities during the selection process.