Senior ML Engineer, Python

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Develop, enhance, and utilize ML Engineering platforms and components; design and implement scalable backend systems, APIs, and microservices using FastAPI.

• Execute MLOps practices including model KPI measurement, tracking, model drift identification, and model feedback loops.

• Deploy and operationalize ML and Deep Learning models, emphasizing LLMs and Generative AI.

• Integrate Azure OpenAI (GPT-4, GPT-4 Vision) and other LLM providers with appropriate retry mechanisms and error management.

• Stay informed about cutting-edge technologies such as LLMs, GenAI, and transformer architectures.

• Scale machine learning algorithms to process large datasets while adhering to strict SLAs.

• Design and orchestrate model pipelines that encompass feature engineering, inference, and ongoing model training.

• Write backend application code in Python and SQL, applying strong object-oriented principles and asynchronous programming (asyncio, async/await).

• Implement dependency injection patterns and a layered architecture (Service, Foundation, Orchestration, DAL).

• Create LLM observability tools (e.g., Langfuse) to monitor prompts, tokens, costs, and latency.

• Develop prompt management systems featuring version control and fallback mechanisms.

• Implement workflows using Celery (or similar) for asynchronous task processing and intricate pipelines.

• Construct multi-tenant architectures ensuring client data isolation.

• Apply cost optimization techniques for LLM utilization (prompt caching, batch processing, token optimization).

• Integrate third-party APIs and services (e.g., document/OCR services, cloud storage, enterprise systems).

• Collaborate with client-facing teams to grasp business context and assist in technical requirement gathering.

• Produce production-ready code that is testable, maintainable, and considers edge cases and errors.

• Maintain high-quality deliverables by adhering to architecture/design guidelines, coding best practices, and conducting periodic design/code reviews.

• Create unit tests and higher-level tests to effectively manage expected edge cases and errors.

• Troubleshoot backend application code using structured logging and distributed tracing.

• Utilize bug tracking, code review, version control, and other tools to organize and deliver work efficiently.

• Engage in scrum calls and agile ceremonies, communicating progress, challenges, and dependencies.

• Document application modifications and updates, including API documentation through OpenAPI/Swagger.

• Investigate and assess emerging architectural patterns and technologies through rapid learning, proofs-of-concept, and prototypes.

⛳️ Requirements

• Bachelor’s or Master’s degree in Computer Science or a related discipline.

• Proficient Python programming skills with 7+ years of experience.

• At least 2 years of practical experience in machine learning and production LLM systems.

• Experience in building backend APIs with FastAPI, async patterns, rate limiting, and SQLAlchemy for 3+ years.

• Proven ability to design maintainable and extensible systems utilizing dependency injection, interfaces, and abstract base classes.

• Familiarity with vector databases such as Pinecone, Weaviate, or Chroma, as well as hybrid search methodologies.

• Strong comprehension of RAG architectures, encompassing retrieval, reranking, context assembly, and response generation.

• Practical experience with LangChain and LangGraph for constructing and managing LLM workflows.

• Advanced Python expertise, including async/await, type hints, Pydantic, and SOLID principles.

• MLOps experience with MLflow, model versioning, and A/B testing; familiarity with Langfuse is a bonus.

• Experience in Natural Language Processing (NLP) and computer vision, including document understanding, OCR, and GPT-4 Vision.

• Experience in building feature pipelines, real-time and batch inference systems, and model serving.

• Hands-on experience with Hugging Face is required; experience with LlamaIndex is advantageous.

• Knowledge of database technologies such as SQL.

• Strong problem-solving abilities and capacity to work in a fast-paced, team-oriented environment.

🏝️ Benefits

• Exciting projects that make a difference.

• Access to Udemy courses of your choice.

• Team-building activities, events, marathons, and charitable initiatives to connect and recharge.

• Workshops, training sessions, and expert knowledge-sharing opportunities that support your growth.

• Defined career progression.

• Absence days to promote work-life balance.

• Flexible hours and work setup - the freedom to work from anywhere and organize your day as you prefer.