AI Benchmark Engineer, Native Language Specialist – Spanish

This is a fully remote position, open to applicants in Mexico.

📋 Description

• Design, develop, and validate Terminal-Bench tasks for multilingual software challenges.

• Assess coding agents.

• Construct realistic task environments utilizing datasets and files in Spanish.

• Generate prompts and pinpoint AI failure points in Spanish.

• Assist in creating robust reference implementations.

• Develop dependable, deterministic verifier scripts.

• Examine execution logs and adjust task difficulty from Easy to Very Hard.

• Execute standard Terminal-Bench configurations on Haiku, Sonnet, and Opus model tiers.

• Engage in a four-layer human quality-control process encompassing creation, human review, calibration review, and audit.

• Collaborate with automated LLM-based checks to guarantee fairness, grammatical precision, and benchmark integrity.

⛳️ Requirements

• Over 5 years of industry experience in software engineering.

• Demonstrated success at leading technology firms and/or graduation from top-tier engineering universities.

• Native or near-native proficiency in Spanish, with a profound understanding of grammar, register, and phrasing conventions.

• High proficiency in English.

• Strong command of Python.

• Proficient in standard shell scripting.

• Expertise in data processing.

• Extensive experience with Terminal/CLI-based development workflows.

• Familiarity with coding agents.

• In-depth technical knowledge of multilingual text processing challenges.

• Experience with encoding/decoding robustness and Unicode normalization.

• Understanding of locale-dependent conventions, such as collation, casing, and non-Gregorian dates.

• Familiarity with text I/O, toolchain interoperability, and safe string operations.

• For specific languages, knowledge of bidirectional/RTL handling, font fallbacks, and rendering/typography in UI or artifacts.

• Reliable availability and commitment; most tasks necessitate a minimum of 2 hours per day or 10 hours per week.

• Ability to submit an updated CV in English.

• Completion of a GenAI assessment, onboarding, and profile setup is required.

• Contractors cannot be engaged in regions subject to international embargoes or sanctions.

• Responsible for personal tax obligations.

🏝️ Benefits

• Flexible schedule as an independent contractor.

• Work at your convenience, as much or as little as you prefer.

• No set hours, no check-ins, and no micromanagement.

• Competitive rates and timely payments.

• Access to a variety of innovative projects.

• Opportunities for portfolio-building and professional development.

• Join a global community of linguists, subject matter experts, and language professionals.

• Potential for supplemental income.

• Note: Contractors do not receive health insurance, paid time off, or retirement contributions.