Data Engineer

This is a fully remote position, open to applicants in Poland.

📋 Description

• Design, implement, and manage MCP and OCR integrations, ensuring the platform's overall stability and performance.

• Concentrate on creating enrichment integrations for LLM prompt generation.

• Enhance platform reliability through performance tuning, database connection management, LLM gateway operations, observability, and autoscaling.

⛳️ Requirements

• Develop and implement secure, resilient integrations with multiple MCP servers to retrieve and consolidate customer data (API clients, batching, connection pooling, retries, rate limiting, caching) and provide enriched context for LLM prompt generation.

• Create and manage OCR integrations for processing attachments (PDFs, images), including tool/API selection, preprocessing, quality management, and text integration into mail classification and prompt pipelines.

• Ensure secure processing of customer and PII data utilized in LLM workflows.

• Scale and optimize the performance of GenAI Python microservices operating in Kubernetes.

• Enhance PostgreSQL session and connection management, including the identification and resolution of connection leaks.

• Manage and secure the LLM-Gateway (LiteLLM), implementing necessary load balancing, throttling, and fallback strategies for GenAI mail processing workflows.

• Establish and maintain observability for enrichment and platform workflows (metrics, traces, structured logs) and contribute to runbooks.

• Optional involvement in the on-call rotation.

• Practical experience with Kubernetes, Helm, and Docker.

• Significant experience in integrating backend APIs, including authentication (tokens/OAuth/OIDC), retries, rate limiting, and caching.

• Hands-on experience with OCR tools/APIs or document processing pipelines.

• Proficient Python skills for integration and troubleshooting (FastAPI, Django, Pydantic).

• Familiarity with PostgreSQL connection management and basic tuning techniques.

• Experience with observability tools (Prometheus, Grafana, ELK).

• Strong understanding of data privacy and handling of PII in production environments.

🏝️ Benefits

• Attractive financial benefits: A cafeteria system that allows employees to customize their benefits by selecting from a range of options.

• Generous referral bonuses, offering up to PLN6,000 for referring specialists.

• Additional revenue sharing opportunities for initiating partnerships with new clients.

• Professional development and team support: Continuous guidance from a dedicated Team Manager for each employee.

• Customized technical mentoring from an assigned technical leader, tailored to individual expertise and project requirements.

• Community and Well-Being: A dedicated team-building budget for both online and on-site team events.

• Opportunities to engage in charitable initiatives and local sports programs.

• A supportive and inclusive work culture that emphasizes diversity and mutual respect.