Python Engineer – AI Data & Evaluation

This is a fully remote position, open to applicants in Brazil, +5 more countries.

📋 Description

• Create and enhance evaluation rubrics and criteria for preference data.

• Analyze and categorize data while uncovering subtle quality concerns and inconsistencies.

• Develop and sustain Python or TypeScript pipelines and infrastructure that facilitate data generation, collection, and evaluation.

• Establish processes that convert qualitative assessments into scalable evaluation workflows.

• Consolidate insights from data reviews into clear technical documentation, updates, and recommendations.

• Work collaboratively with researchers and engineers to enhance evaluation methodologies and data quality.

• Revise evaluation processes based on identified issues and project demands.

• Present technical findings in a structured, concise, and actionable manner.

⛳️ Requirements

• Minimum of 5 years of professional experience with Python or TypeScript.

• Hands-on experience in building or maintaining pipelines and infrastructure using Python, or contributing to extensive TypeScript codebases.

• Practical knowledge in preference data collection, review, or rubric development.

• Experience in designing or implementing evaluation criteria for AI-generated content or technical outputs.

• Excellent written and verbal communication skills, evidenced by clear technical documentation or summaries.

• Capability to articulate and justify quality judgments using explicit evaluation criteria.

• Experience in navigating or contributing to large codebases, such as VSCode, is a plus.

• Previous experience in AI, machine learning, data quality, or evaluation projects is advantageous.

• Essential experience working in Tier 1 companies.

🏝️ Benefits

• Opportunity to work on cutting-edge technology in a dynamic environment.

• Collaborative team culture that values innovation and creativity.

• Competitive salary and comprehensive benefits package.

• Continuous professional development and career growth opportunities.