AI Evaluator

This is a fully remote position, open to applicants in Kansas.

📋 Description

• Evaluating and assessing the performance of AI models or algorithms based on their outputs or behaviors through a series of evaluative questions.

• Labeling specific elements within a piece of content rather than assessing the entire content itself.

• Assigning predefined categories or tags to various items.

• Assessing the perceived quality and/or appropriateness of content.

• Generating labels to enhance the understanding of concepts, trends, etc.

• Creating supplementary training data for machine learning models by applying transformations to the original data, such as image modifications (rotation, flipping, cropping), generating new text (paraphrasing, summarization), or altering audio/video signals (speed adjustments, pitch changes) to mitigate overfitting and enhance dataset diversity.

• Reviewing data to determine whether a product feature functions as intended according to the project's guidelines.

• Labeling model outputs to indicate whether a piece of content meets specific criteria. Examples include identifying clickbait, gaming videos, or branded content.

• Organizing or ranking items based on a defined set of preferences or criteria.

• Formulating prompts or questions that will elicit responses from a language model or other AI systems.

• Conducting projects that assess content relevance based on a relevancy scale (1-3, 1-5, etc.).

• Generating responses to prompts or questions utilizing a language model or other AI systems.

• Rewriting existing text while maintaining the original meaning, often to enhance clarity or style and ensure adherence to guidelines.

• Producing concise summaries of longer texts or data sets.

• Transcribing spoken language or audio content into written text.

• Translating text or spoken language from one language to another.

• Collecting and compiling diverse forms of data to be utilized for training, evaluating, or fine-tuning AI models. This may encompass text, images, videos, audio files, or other types of digital content.

⛳️ Requirements

• A Bachelor’s degree or higher in a humanities-related field is mandatory. Advanced degrees (Master’s or PhD) are highly preferred.

• Professional or expert-level proficiency (C1/C2) in English.

🏝️ Benefits

• Opportunities for professional development and growth.

• A collaborative and innovative work environment.

• Competitive salary and benefits package.