Senior Data Labeling Specialist (Boston)

Senior Data Labeling Jobs in Boston (Remote, Full-Time) Rex.zone is hiring senior data labeling professionals to lead training data quality for AI/ML systems. This role combines data labeling, RLHF evaluation, prompt evaluation, and QA to improve large language model performance across end-to-end LLM training pipelines.

What You Will Do

• Lead complex data labeling and QA evaluation for LLM training pipelines
• Execute RLHF-style preference ranking and rubric-based prompt evaluation
• Create, refine, and enforce annotation guidelines; drive annotator calibration
• Perform named entity recognition and taxonomy-based labeling for NLP datasets
• Support computer vision annotation workflows (bounding boxes, polygons, keypoints) as needed
• Run quality audits, disagreement analysis, and error taxonomy reporting
• Flag ambiguous edge cases and collaborate with engineering and ops on policy updates
• Maintain documentation for annotation guidelines compliance and audit readiness
• Track quality metrics (accuracy, inter-annotator agreement) and recommend process changes

Required Qualifications

• Experience leading or mentoring labelers in production annotation programs
• Strong understanding of training data quality, quality assurance, and evaluation rubrics
• Hands-on experience with RLHF-style ranking, LLM evaluation, or prompt evaluation
• Familiarity with named entity recognition and structured labeling schemas
• Ability to document decisions clearly and apply policy consistently
• Excellent attention to detail and ability to deliver high accuracy at scale

Work Model Remote role within the US, aligned to Boston-area schedules and availability.

Includes scheduled calibration and QA review sessions to maintain consistent quality.

Compensation Hourly base pay: $30–$50 (USD), based on scope, domain complexity, and demonstrated quality leadership.

FAQ Highlights

• Remote? Yes—work is performed remotely within the US, with Boston alignment.
• What is “senior data labeling”? Owning training data quality: QA evaluation, calibration, guideline compliance, and edge case resolution.
• Task types? RLHF preference ranking, prompt evaluation, NER, content safety labeling, and (as needed) computer vision annotation.