Senior Data Annotator (Amsterdam)

Senior Data Annotator (Amsterdam) — Remote (US) Rex.zone is hiring a Senior Data Annotator to create high-quality labeled datasets and evaluation judgments that power AI/ML training pipelines across NLP, computer vision, content safety, and LLM evaluation. You will perform data labeling, RLHF preference ranking, prompt evaluation, and QA evaluation to improve training data quality, annotation guidelines compliance, and model performance.

What You Will Do

• Produce high-accuracy annotations for NLP and computer vision datasets
• Perform RLHF preference ranking, pairwise comparisons, and rubric-based scoring
• Conduct prompt evaluation and response grading for helpfulness, correctness, and safety
• Execute QA evaluation on sampled batches to measure training data quality and audit readiness
• Apply named entity recognition (NER), taxonomy tagging, and structured labeling
• Label content safety categories (policy, hate, harassment, self-harm, sexual content) with calibrated severity
• Write clear rationales for difficult cases and document edge-case decisions
• Maintain and improve annotation guidelines; flag schema drift, label leakage, and ambiguous definitions

Required Qualifications

• Experience with data annotation/data labeling in production workflows
• Strong attention to detail and ability to follow complex guidelines
• Ability to perform QA evaluation and explain labeling decisions clearly
• Familiarity with RLHF concepts, preference data, and evaluation rubrics
• Self-directed and reliable throughput in a remote environment

Preferred Qualifications

• Experience with NER schemas and ontology/taxonomy design
• Hands-on work with computer vision annotation tools (boxes, polygons, keypoints, segmentation)
• Content safety labeling and policy interpretation experience
• Experience with golden sets, audits, and inter-annotator agreement processes

Work Arrangement Remote, FULL_TIME. While the keyword targets Amsterdam-based searches, the role is hired under US country settings and supports distributed teams.

Compensation $30–$50 per hour (base salary, hourly).