AI Benchmark Engineer, Native Language Specialist – Chinese

This is a fully remote position, open to applicants in Hong Kong.

📋 Description

• Design, develop, and assess Terminal-Bench tasks for multilingual software challenges.

• Evaluate coding agents.

• Create realistic task environments utilizing datasets and files in Chinese.

• Maintain benchmark assets in the target language to evaluate multilingual capabilities.

• Identify AI failure points via prompting and translation in the native language.

• Provide support for robust reference implementations.

• Develop reliable, deterministic verifier scripts.

• Analyze execution logs and adjust task difficulty from Easy to Very Hard.

• Execute standard Terminal-Bench configurations against Haiku, Sonnet, and Opus model tiers.

• Engage in a four-layer human quality-control process that includes creation, human review, calibration review, and audit.

• Collaborate with automated LLM-based checks to ensure fairness, grammatical accuracy, and benchmark integrity.

⛳️ Requirements

• Over 5 years of industry experience in software engineering.

• Demonstrated success at leading technology firms and/or graduation from top-tier engineering institutions.

• Native or near-native proficiency in Chinese, with a comprehensive understanding of grammar, register, and phrasing rules.

• High proficiency in English.

• Strong skills in Python, standard shell scripting, and data processing.

• Extensive experience with Terminal/CLI-based development workflows.

• Familiarity with coding agents.

• Deep technical knowledge of multilingual text processing challenges.

• Understanding of encoding/decoding robustness and Unicode normalization.

• Awareness of locale-dependent conventions, including collation, casing, and non-Gregorian dates.

• Knowledge of text I/O, toolchain interoperability, and safe string operations.

• For applicable languages, familiarity with bidirectional/RTL handling, font fallbacks, and rendering/typography in UI or artifacts.

• Reliable availability and commitment.

• Ability to complete assigned tasks within deadlines.

• Most tasks necessitate availability of at least 2 hours per day or 15–20 hours per week.

• CV must be submitted in English.

• Geographic restrictions may be applicable due to international embargoes or sanctions.

🏝️ Benefits

• Flexible independent contractor schedule without fixed hours, check-ins, or micromanagement.

• Competitive pay rates.

• Prompt payment processing.

• Access to a variety of innovative projects.

• Opportunities for portfolio and skills enhancement across diverse industries and domains.

• Flexibility to work from anywhere at any time.

• Global network of linguists, subject matter experts, and language professionals.

• Note: No health insurance, paid time off, or retirement contributions are provided.

• Hours are not guaranteed.

• Supplemental-income opportunity with variable work availability.