ML Annotation QA Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Take ownership of quality analysis that requires significant judgment on annotated data related to computer vision and machine learning.

• Review daily queues of annotations and generate in-house evaluations along with root-cause analyses.

• Manage, version, and enhance verdict taxonomies, decision-making rules, and quality standards.

• Create and sustain performance trackers that monitor error rates by facility, site, equipment, and data format over time.

• Identify baseline anomalies and raise flags promptly.

• Differentiate between annotation errors, model or system errors, and actual field degradation.

• Present findings to engineering and machine learning teams with reproducible evidence and confidence levels stated.

• Recognize systematic failure patterns and keep a documented library of these patterns.

• Query and analyze annotation data through Python and SQL.

• Integrate findings into standard operating procedures and revisions of annotation instructions.

• Specify and validate enhancements to annotation tools.

• Establish quality analysis and reporting mechanisms for new annotation initiatives.

• Document work in Jira and contribute to pre-release validations.

• Initially assist with warehouse forklift vision, barcode readability, and localization assessments.

• Collaborate with Machine Learning Engineers, Quality Assurance, and Engineering teams.

⛳️ Requirements

• Bachelor's degree in Computer Science/Engineering, Electrical Engineering, or equivalent professional experience.

• 2 to 5 years of experience in machine learning quality assurance, annotation quality, data quality, or analytics within an AI/ML organization.

• Practical experience with annotated machine learning datasets, particularly in evaluating label quality.

• Proven ability to identify, diagnose, and communicate data anomalies to a technical audience.

• Strong grasp of statistics and competence in data manipulation.

• Skills in root cause analysis, including the capacity to generate competing hypotheses and pinpoint separating evidence.

• Familiarity with Python and SQL, or similar tools, for independent data querying and analysis.

• Experience in drafting quality guidelines, decision-making rules, and labeling taxonomies.

• Knowledge of data privacy and confidentiality regulations concerning customer operational data.

• Exceptional documentation, communication, and collaboration abilities.

• Capability to work autonomously with unfamiliar datasets.

• Experience with Jira.

• Preferred experience in authoring annotation guidelines or standard operating procedures.

• Preferred background in warehouse automation, robotics, or computer vision applications.

🏝️ Benefits

• Comprehensive health and wellness benefits.

• Opportunities for professional development and growth.

• Collaborative and innovative work environment.

• Flexible work arrangements.

• Competitive salary and performance-based incentives.