Mid/Senior Data Engineer – Databricks

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Convert business challenges and opportunities into solutions utilizing Artificial Intelligence and Machine Learning.

• Identify high-impact use cases and assess the financial, operational, and strategic effects of the models created.

• Design, construct, validate, and enhance sophisticated end-to-end Machine Learning models.

• Investigate and prepare data, deploy models, and oversee them in production settings.

• Apply MLOps methodologies for automating pipelines, managing model versions, monitoring performance, and detecting drift.

• Design and refine scalable data and Machine Learning architectures within cloud environments, particularly GCP.

• Ensure the solutions' performance, security, availability, and operational effectiveness.

• Implement best practices in development, data governance, observability, documentation, and lifecycle management of models.

• Act as a technical resource for Data and Artificial Intelligence projects.

• Assist in decision-making, establish architectural standards, and disseminate knowledge among multidisciplinary teams.

• Assess new technologies, frameworks, and methodologies related to Artificial Intelligence, Machine Learning, and Generative AI.

⛳️ Requirements

• High-level expertise in Python or R for creating analytical models, managing data, automating processes, and deploying Machine Learning solutions in production settings.

• Strong experience with advanced SQL for extracting, transforming, cleaning, and integrating large datasets.

• Demonstrated experience in building, training, validating, optimizing, and deploying both supervised and unsupervised models.

• Familiarity with Scikit-Learn, XGBoost, LightGBM, and similar libraries.

• Comprehensive experience in the complete Machine Learning model development lifecycle, including versioning, monitoring, reproducibility, and pipeline automation.

• Proficiency with Google Cloud Platform (GCP), Spark, Databricks, and BigQuery.

• In-depth knowledge of statistics, statistical inference, mathematical modeling, and experimental design.

• Experience in conducting and interpreting A/B testing.

• Capability to translate analytical results and performance metrics into actionable insights.

• Experience in developing and monitoring business indicators and model metrics.

• Understanding of creating scalable architectures for data processing, storage, and consumption.

• Ability to operate in highly complex analytical settings.

• Background in the healthcare industry.

🏝️ Benefits

• No benefits, perks, or additional compensation elements are explicitly stated in the posting.