AI Benchmark Engineer, Native Language Specialist – French

This is a fully remote position, open to applicants in Belgium.

📋 Description

• Design, construct, and verify Terminal-Bench tasks for multilingual software challenges.

• Assess coding agents.

• Create realistic task environments utilizing datasets and files in French.

• Detect AI failure points through prompting and translation in the target language.

• Provide support for robust reference implementations.

• Develop reliable, deterministic verifier scripts.

• Analyze execution logs and adjust task difficulty from Easy to Very Hard.

• Execute standard Terminal-Bench configurations against Haiku, Sonnet, and Opus model tiers.

• Engage in a four-layer human quality control process that includes creation, human review, calibration review, and audit.

• Collaborate with automated LLM-based checks to guarantee fairness, grammatical precision, and benchmark integrity.

⛳️ Requirements

• Over 5 years of professional experience in software engineering.

• Established history at leading technology firms and/or a degree from prestigious engineering universities.

• Native or near-native proficiency in French, with a comprehensive understanding of grammar, register, and phrasing rules.

• High proficiency in English.

• Strong skills in Python, standard shell scripting, and data processing.

• Extensive experience with Terminal/CLI-based development workflows.

• Familiarity with coding agents.

• In-depth technical knowledge of multilingual text processing challenges.

• Understanding of encoding/decoding robustness and Unicode normalization.

• Familiarity with locale-dependent conventions, such as collation, casing, and non-Gregorian dates.

• Knowledge of text I/O, toolchain interoperability, and safe string operations.

• Application must include an updated CV in English.

• Completion of a GenAI assessment is required.

• Reliable availability and commitment; most tasks necessitate a minimum of 2 hours per day or 10 hours per week.

• Contractors are prohibited from working in regions subject to international embargoes or sanctions.

• Contractors are responsible for their own tax liabilities.

🏝️ Benefits

• Flexible schedule with the freedom to work at your convenience.

• No fixed hours, check-ins, or micromanagement.

• Competitive compensation rates.

• Timely payments.

• Access to a variety of innovative projects.

• Opportunities for portfolio and skills enhancement.

• Membership in a global community.

• No employer-sponsored health insurance, paid time off, or retirement contributions.

• Opportunity for supplemental income with varying project availability.