Artificial Intelligence Analyst, Mid-Level

atInstituto Hardware BR HBRRemote

Brazil Full-time Threat Intelligence SpecialistMid-levelSenior

Posted 1 day ago

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design, develop, and improve comprehensive RAG pipelines, which encompass data ingestion, chunking, tokenization, embeddings, semantic/hybrid search, reranking, and response generation.

• Create ETL/ELT processes for both structured and unstructured data sources, incorporating incremental updates, quality validation, and the anonymization/masking of sensitive information.

• Oversee the generation, storage, updating, and versioning of embeddings, ensuring integration with vector, relational, and NoSQL databases.

• Conduct ongoing evaluations of LLMs and RAG systems, assessing retrieval efficiency, groundedness, hallucinations, latency, and cost-effectiveness.

• Track quality, latency, availability, cost, and resource utilization in production environments; automate deployment, versioning, and rollback through CI/CD practices.

• Manage versioning of prompts, models, datasets, embeddings, and pipelines, ensuring data lineage, auditing, and observability.

• Develop versioned prompts, system prompts, structured outputs, and tool invocation.

• Provide services via REST/gRPC APIs and microservices operating on Docker and Kubernetes.

⛳️ Requirements

• Bachelor's degree in Computer Science/Engineering, Software Engineering, Data Science, Information Systems, Electrical Engineering, or a related discipline.

• Hands-on experience in building and maintaining LLM-based applications and RAG architectures within production settings.

• Proficiency in Python and SQL; familiarity with frameworks such as Hugging Face Transformers, PyTorch, or similar technologies; knowledge in NLP and embeddings.

• Experience with vector databases, semantic search, SQL/NoSQL databases, and ETL/ELT processes.

• Knowledge of MLOps/LLMOps, Git, CI/CD, Docker, Kubernetes, and at least one cloud platform (AWS, GCP, or Azure).

• Understanding of data governance, data lineage, anonymization, authentication/authorization, and compliance with Brazil’s LGPD data protection legislation.

• Intermediate proficiency in English (sufficient for reading technical documentation and scientific literature).