RLHF Quality Analyst

About the role

You will judge what a language model produces - ranking responses, applying safety policy and writing the preference data that model training runs on. The work is careful reading and consistent judgement, at volume, against a rubric that keeps moving as the policy sharpens.

What you will do

• Review and rank model responses for helpfulness, accuracy and safety against a written rubric.
• Apply trust safety policy to borderline content and record why a call was made, not only what it was.
• Flag rubric gaps and recurring ambiguity so the guideline gets fixed rather than worked around.
• Take part in calibration sessions and golden-set checks that keep judgement consistent across the team.
• Maintain the quality bar under deadline without drifting from the rubric.

What you will bring

• Strong reading comprehension and written English - this is the core skill of the role.
• Evident care with detail and the patience for repetitive, judgement-heavy work.
• Willingness to have your judgement measured, discussed and corrected in calibration.
• A bachelors degree in any discipline, or equivalent experience. This role is open to candidates early in their career; the training is provided.
• Helpful, not required: content moderation, editorial, teaching, linguistics or translation experience, or a second Indian language.

Working arrangements

Full time, on site at the delivery centre listed above. Standard working hours, with shift cover arranged within the team where a client programme runs to a different timezone.

About DesiCrew

DesiCrew has run high-stakes AI and enterprise operations from small-town and rural India since 2007. More than 2,000 people work across our delivery centres, 68% of them women - roughly twice the industry average - and 49% the sole earner in their family. We are ISO 9001, ISO 27001 and ISO 42001 certified and a Great Place to Work.

We hire for aptitude and train for the rest. DTOUCH, our in-house learning platform, has delivered more than 5,000 certifications and 1,200 AI-certified professionals, and training continues well past onboarding.

Send us your CV and tell us why this role. We read every application.

Disclaimer: This job description has been sourced from a public domain and may have been modified by Naukri.com to improve clarity for our users. We encourage job seekers to verify all details directly with the employer via their official channels before applying.