ML Engineer - Language Models & Evaluations

About Deccan AI

Deccan AI is a fast-growing, venture-backed AI infrastructure company focused on training, evaluating, and improving next-generation AI systems. Headquartered in the Bay Area, with a growing India hub in Hyderabad, the company was founded by alumni of IIT Bombay, IIM Ahmedabad, and former Google leaders.

We work with some of the worlds leading AI frontier labs and research organizations, including Google DeepMind, Snowflake, and other cutting-edge AI teams. Backed by Prosus Ventures, Deccan AI recently raised $25M in Series A funding and is entering a significant growth phase.

With a global network of over 1 million experts, advanced automation systems, and vertically integrated platforms, we deliver the high-quality data and evaluation infrastructure that state-of-the-art AI models depend on. As the AI infrastructure market rapidly expands, Deccan AI is building the systems powering the future of AI.

About the Role

Were hiring an ML Engineer to work on Language Models & Evaluations, partnering directly with a frontier AI lab. Youll be building and stress-testing the systems behind next-generation LLMs not maintaining internal tooling, but working on problems at the actual frontier of whats being built in AI today.

What Youll Do

• Fine-tune and adapt large language models for specific tasks and domains
• Design and build evaluation frameworks, benchmarks, and automated scoring pipelines
• Build red-teaming and adversarial testing pipelines to surface model failure modes
• Work on RLHF, instruction tuning, and alignment techniques
• Analyze model outputs for hallucination, bias, and safety issues, and feed insights back into training

What Were Looking For

• 2-4 years of hands-on ML/NLP experience, with real exposure to LLMs
• Strong Python skills and experience with PyTorch or similar frameworks
• Familiarity with Hugging Face Transformers, LangChain/LlamaIndex, vLLM, DeepSpeed, LoRA/QLoRA, or PEFT
• Understanding of evaluation methodology not just building models, but knowing how to measure if theyre actually good
• Bonus: experience with RLHF pipelines, published research, or open-source LLM contributions

Why Join

• Direct exposure to frontier-lab-scale LLM problems
• Work alongside top ML research and engineering talent
• High-ownership role with real technical depth

Disclaimer: This job description has been sourced from a public domain and may have been modified by Naukri.com to improve clarity for our users. We encourage job seekers to verify all details directly with the employer via their official channels before applying.