AI Benchmark Engineer, Native Language Specialist – Chinese

This is a fully remote position, open to applicants in Taiwan.

📋 Description

• Create, develop, and verify Terminal-Bench tasks for multilingual software solutions.

• Assess coding agents.

• Construct realistic task settings utilizing datasets and files in Chinese (Taiwan).

• Identify AI failure points through prompting and translation in the native language.

• Produce robust reference implementations.

• Develop dependable, deterministic verifier scripts.

• Examine execution logs and adjust task difficulty ranging from Easy to Very Hard.

• Execute standard Terminal-Bench configurations against Haiku, Sonnet, and Opus model tiers.

• Engage in a comprehensive four-layer human quality-control process that includes creation, human review, calibration review, and audit.

• Collaborate with automated LLM-based checks to ensure fairness, grammatical precision, and benchmark integrity.

⛳️ Requirements

• Over 5 years of industry experience in software engineering.

• Proven history at leading technology firms and/or graduation from prestigious engineering universities.

• Native or near-native proficiency in Chinese (Taiwan), with a profound understanding of grammar, register, and phrasing rules.

• High proficiency in English.

• Strong expertise in Python.

• Excellent skills in standard shell scripting.

• High proficiency in data processing.

• Extensive experience with Terminal/CLI-based development workflows.

• Familiarity with coding agents.

• In-depth technical knowledge of multilingual text processing challenges.

• Experience with encoding/decoding robustness and Unicode normalization.

• Knowledge of locale-dependent conventions, including collation, casing, and non-Gregorian dates.

• Experience with text I/O, toolchain interoperability, and safe string operations.

• An updated CV in English.

• Successful completion of a GenAI assessment.

• Reliable availability and commitment; most tasks require a minimum of 2 hours per day or 15–20 hours per week.

• Contractors must not reside in regions subject to international embargoes or sanctions.

• Contractors are responsible for their own tax obligations.

🏝️ Benefits

• Flexible schedule without fixed hours, check-ins, or micromanagement.

• Competitive compensation rates.

• Timely payments.

• Access to a variety of innovative projects.

• Opportunities for portfolio and skills enhancement across different industries and domains.

• Global network of linguists, subject matter experts, and language professionals.

• No health insurance, paid time off, or retirement contributions are offered.

• Hours are not guaranteed.

• Supplemental-income opportunity for independent 1099 contractors.