AI Engineer

This is a fully remote position, open to applicants in Germany.

📋 Description

• Advanced Prompt Engineering: Crafting intricate and dynamic prompt templates that incorporate conditional logic, while effectively reusing information and context to enhance generation quality and reasoning capabilities.

• Structured Outputs & Schemas: Deploying a variety of response formats (such as JSON mode, function calls, and Zod/JSON schemas) to guarantee that AI outputs are predictable and seamlessly integrable into application logic.

• Prompt Engineering & Evaluations: Establishing comprehensive evaluation pipelines and leveraging Langfuse to gather feedback and assess the quality of responses in real time.

• Tracing & Debugging: Conducting thorough debugging of complex LLM chains utilizing Langfuse traces to pinpoint bottlenecks and optimize cost, latency, and context window usage.

• AI A/B Testing: Executing systematic experiments across different models via OpenRouter (e.g., comparing Claude 3.5 Sonnet to GPT-4o) and analyzing outcomes based on quantitative metrics.

• Data-Driven Decisions: Making deployment decisions for new prompts or models strictly based on quantitative benchmarks and trace data, rather than relying on intuition.

• Output Scoring & Analysis: Creating scoring systems to evaluate the “Problem → Solution” chain and identify the root causes of hallucinations or logical errors using Langfuse analytics.

• Model Performance & Fine-Tuning: Regularly reassessing model performance as new architectures are released and conducting fine-tuning as required to address specific domain needs.

⛳️ Requirements

• Node.js & Next.js: Extensive expertise in this stack to develop reliable services and effectively manage complex LLM-generated data.

• Dynamic Prompting Skills: Demonstrated experience in crafting prompts that are highly influenced by input variables and context injection.

• OpenRouter Experience: Familiarity with unified APIs, managing rate limits, and selecting the most cost-efficient models for particular tasks.

• Langfuse (or similar): Knowledge of LLM observability principles, including setting up tracing, creating test datasets, and integrating scoring systems.

• Evaluation Methodology: Experience with frameworks like RAGAS or the development of custom “LLM-as-a-judge” systems.

• Analytical Mindset: Capability to convert raw generation logs into actionable business metrics and technical insights.

• Iterative Mindset: Commitment to ongoing product enhancement through continuous feedback loops.

🏝️ Benefits

• Remote Work Environment: Enjoy the flexibility to work from any location at any time, fostering a healthy work-life balance.

• Unlimited PTO: Take advantage of unlimited paid time off to recharge and prioritize your well-being, without the constraints of counting days.

• Paid National Holidays: Relax and celebrate national holidays with paid time off to unwind and rejuvenate.

• Company-provided MacBook: Experience optimal productivity with high-quality Apple MacBooks provided to all employees who require them.

• Flexible Independent Contractor Agreement: Benefit from flexibility, autonomy, and entrepreneurial opportunities, along with tax advantages, networking opportunities, reduced employment obligations, and the freedom to work from anywhere.