AI Benchmark Engineer, Native Language Specialist – German

This is a fully remote position, open to applicants in Switzerland.

📋 Description

• Design, develop, and validate Terminal-Bench tasks addressing multilingual software challenges.

• Assess coding agents.

• Create realistic task environments utilizing datasets and files in German while maintaining assets in the target language.

• Identify AI failure points through prompting and translation in the native language.

• Develop robust reference implementations.

• Write dependable and deterministic verifier scripts.

• Analyze execution logs and adjust task difficulty from Easy to Very Hard using Terminal-Bench configurations and Haiku, Sonnet, and Opus model tiers.

• Engage in a four-layer human quality control process alongside automated LLM-based checks.

• Guarantee fairness, grammatical accuracy, and benchmark integrity.

⛳️ Requirements

• A minimum of 1 year of industry experience in software or prompt engineering.

• Demonstrated success at leading technology firms and/or a degree from prestigious engineering universities.

• Native or near-native proficiency in German, with a comprehensive understanding of grammar, register, and phrasing rules.

• High proficiency in English.

• Strong skills in Python, standard shell scripting, and data processing.

• Extensive experience with Terminal/CLI-based development workflows.

• Familiarity with coding agents.

• In-depth technical knowledge of multilingual text processing challenges.

• Understanding of encoding/decoding robustness and Unicode normalization.

• Familiarity with locale-dependent conventions, including collation, casing, and non-Gregorian dates.

• Knowledge of text I/O, toolchain interoperability, and secure string operations.

• Up-to-date CV in English.

• Successful completion of a GenAI assessment.

• Reliable availability and commitment; most tasks necessitate a minimum of 2 hours per day or 10 hours per week.

🏝️ Benefits

• Flexible scheduling with no fixed hours, check-ins, or micromanagement.

• Timely payments.

• Access to a variety of innovative projects.

• Opportunities for portfolio-building and professional development.

• A global community of linguists, subject matter experts, and language professionals.

• No benefits such as health insurance, paid time off, or retirement contributions are offered.

• Supplemental income opportunity with variable project availability.