Senior Data Annotator Jobs Prague

Overview Rex.zone is hiring for Senior Data Annotator roles focused on producing and validating high-quality training data for AI/ML systems. You will support LLM training pipelines and AI model evaluation through data labeling, RLHF preference ranking, prompt-response evaluation, and QA evaluation across NLP, computer vision, and content safety datasets.

What You Will Do

• Label and review data for NLP tasks such as text classification, named entity recognition (NER), summarization, and instruction-following
• Perform computer vision annotation including bounding boxes, polygons, keypoints, and segmentation
• Execute content safety labeling using defined policies and rubrics
• Create RLHF signals via preference ranking and rubric-based scoring for LLM evaluation
• Run QA evaluation workflows including audits, calibration, consensus checks, and disagreement resolution
• Document edge cases, provide clear rationales, and flag guideline gaps to improve annotation guidelines compliance
• Track and improve training data quality using accuracy, inter-annotator agreement, and error taxonomy feedback

Requirements

• Mid-Senior experience in data annotation, data labeling, and/or QA evaluation for AI training data
• Strong ability to follow guidelines, handle ambiguity, and justify labeling decisions
• Familiarity with NLP and/or computer vision annotation workflows and schemas
• Experience contributing to training data quality programs (sampling, audits, consistency checks) in distributed teams

Nice to Have

• Experience with RLHF datasets, prompt evaluation, and preference modeling for instruction-tuned LLMs
• Background in content safety labeling and large language model evaluation
• Exposure to calibration sessions, rubric design, and inter-annotator agreement measurement

Workplace This is a Remote , Full-time role supporting distributed collaboration across time zones.

Compensation Competitive hourly base pay: $30–$50 per hour .

How To Apply Apply via Rex.zone and highlight experience in RLHF, prompt evaluation, QA evaluation, NER, computer vision annotation, content safety labeling, and examples of improving training data quality through audits and guideline feedback.