AI Benchmark Engineer, Native Language Specialist – Spanish

This is a fully remote position, open to applicants in Argentina.

📋 Description

• Design, construct, and validate Terminal-Bench benchmark tasks addressing multilingual software challenges.

• Assess coding agents effectively.

• Create realistic task environments utilizing datasets and files in Spanish.

• Generate prompts and translations in the target language to pinpoint AI failure points.

• Develop reliable reference implementations.

• Author dependable, deterministic verifier scripts.

• Examine execution logs and adjust task difficulty levels from Easy to Very Hard.

• Execute standard Terminal-Bench configurations on Haiku, Sonnet, and Opus model tiers.

• Engage in a four-tier human quality-control process: creation, human review, calibration review, and audit.

• Collaborate with automated LLM-based checks to guarantee fairness, grammatical accuracy, and benchmark integrity.

⛳️ Requirements

• Over 5 years of professional experience in software engineering.

• Demonstrated history at leading technology firms and/or graduation from prestigious engineering universities.

• Native or near-native fluency in Spanish, with a comprehensive understanding of grammar, register, and phrasing conventions.

• High proficiency in English.

• Strong skills in Python, standard shell scripting, and data processing.

• Extensive experience with Terminal/CLI-based development workflows.

• Working knowledge of coding agents.

• In-depth technical comprehension of multilingual text processing challenges.

• Familiarity with encoding/decoding robustness and Unicode normalization.

• Understanding of locale-dependent conventions, including collation, casing, and non-Gregorian dates.

• Knowledge of text I/O, toolchain interoperability, and safe string operations.

• For specific languages, understanding of bidirectional/RTL handling, font fallbacks, and rendering/typography in UI or artifacts.

• Reliable availability and commitment; most tasks necessitate a minimum of 2 hours per day or 10 hours per week.

• Ability to provide an updated CV in English.

• Capability to complete a GenAI assessment.

• As a 1099 contractor, responsible for own tax obligations.

• Contractors must not reside in regions subject to international embargoes or sanctions.

🏝️ Benefits

• Flexible schedule with no fixed hours, check-ins, or micromanagement.

• Competitive compensation rates.

• Timely payments.

• Opportunities to work on diverse and innovative projects.

• Portfolio development and skill enhancement.

• Access to a global community of linguists, subject matter experts, and language professionals.

• No health insurance, paid time off, or retirement contributions are offered.

• Opportunity for supplemental income with varying work availability.