Retail SME - AI Evaluation Expert

About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Retail Specialist

Type: Contract Compensation: $60–$80/hour Location: Remote Role Responsibilities

• Guide research and engineering teams to close knowledge gaps in retail merchandising, category management, and operations reasoning.
• Design challenging, domain-relevant retail tasks and write accurate, well-reasoned solutions grounded in real retail practice.
• Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness, judgment, and reasoning quality.
• Develop and refine evaluation guidelines and scoring rubrics specific to retail tasks.
• Collaborate with other subject matter experts to ensure consistency and accuracy in training data.

Qualifications Must-Have

• 8+ years of dedicated professional experience in retail (e.g., merchandising, category management, retail operations, buying/planning) at a recognized, top-tier organization (e.g., Amazon, Walmart, Target, Nike, Costco, Home Depot, or equivalent).
• Prior hands-on experience evaluating LLM/AI model outputs against rubrics or structured scoring criteria — mandatory; please describe this experience in your application.
• Demonstrable career progression (e.g., Category Manager → Senior Manager → Director of Merchandising).
• Ability to engage reliably for at least 35 hours/week during weekdays.
• Verbal and written communication skills, problem-solving skills, and interpersonal skills.

Compensation & Legal

• W-2 employment with Cincinnatus LLC

Application Process (Takes 20–30 mins to complete)

• Upload resume
• AI interview based on your resume
• Submit form

Resources & Support

• For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
• For any help or support, reach out to: [email protected]

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.