LLM Evaluation Engineer

Job Title: LLM Evaluation Engineer

Alternative Titles: AI Evaluation Engineer | LLM Quality Engineer | AI Research Engineer

Location: Bangalore

Employment Type: Full-Time

Experience: 3–5 Years

Job Description

Infyni is hiring an LLM Evaluation Engineer for one of our end clients for a full-time role in Bangalore.

The selected candidate will be employed on the end client's payroll. The client name will remain

confidential during the hiring process.

Required Skills

• Strong hands-on experience with Python and benchmark design.

• Experience building test harnesses, verifiers, and reward signals.

• Expertise in evaluation metrics and assessing LLM performance.

• Familiarity with SWE-bench, LLM-as-judge, pytest, adversarial evaluation, and calibration.

Ideal Candidate: An engineer with 3–5 years of relevant experience designing, implementing, and

improving evaluation frameworks to measure the quality, reliability, and performance of LLMs and AI

systems.

Preferred: Candidates from premier institutes such as IITs or equivalent institutions are most welcome.