AI Agentic Evaluator

Role & responsibilities

• Review agentic traces from AgentX, including tool calls, tool results, and interactions with its data environment, to understand the full sequence of steps the agent took, not just its final answer.
• Read two candidate responses for a given task and select the one that better completes the task, follows instructions, and behaves appropriately along the way.
• Verify each response against the trace: check that numbers, facts, and conclusions match what the tools actually returned, and catch hallucinations, misreadings, or skipped steps.

Preferred candidate profile

Preferred skills:

• Prior experience in AI data labelling, content rating, RLHF, or model evaluation, especially side-by-side (pairwise) comparison tasks.
• Individuals who uses claude code/cursor/codex is preferred.
• Background in linguistics, localisation, QA, data analysis, or related fields.
• Experience working with AI assistants or agent-based tools in a professional or personal capacity.

Interested candidates can share their profile at [email protected]