AI QA Engineer

Agentic AI QA Engineer

Experience: 3+ Years

Location: Delhi NCR / Remote

Notice Period: Immediate Joiners Preferred

About the Role

We are looking for a skilled Agentic AI QA Engineer to join our team working on a cutting-edge Agentic AI project. The ideal candidate will have hands-on experience testing AI agents and agentic workflows, with a strong grasp of how these systems are designed, built, and expected to behave in real-world scenarios. This role is critical to ensuring the reliability, safety, and performance of AI-driven agent systems before they reach production.

Key Responsibilities

• Design and execute test strategies and test cases tailored specifically to agentic AI systems and multi-step autonomous workflows
• Perform functional, integration, and end-to-end testing of AI agents across various use cases and environments
• Identify and document edge cases, failure scenarios, and hallucination risks in AI agent behaviour
• Validate agent decision-making, tool usage, task completion, and multi-turn interaction flows
• Collaborate closely with AI/ML engineers and product teams to understand agent architecture and design comprehensive test coverage
• Build and maintain automated test suites using relevant testing frameworks and tools
• Analyse test results, log defects, and drive issues to closure with development teams
• Contribute to continuous improvement of QA processes for agentic AI systems

Required Skills & Qualifications

• 3+ years of overall QA/testing experience, with hands-on exposure to testing AI agents or agentic workflows
• Strong understanding of Agentic AI concepts how AI agents are architected, function, and interact with tools/environments
• Practical experience with functional, integration, and end-to-end testing methodologies
• Familiarity with testing frameworks and automation tools relevant to AI systems (e.g., Python-based test frameworks, API testing tools, CI/CD pipelines)
• Ability to independently design test strategies and detailed test cases for complex, non-deterministic AI systems
• Strong analytical skills to identify edge cases, failure modes, and hallucination risks in LLM/agent outputs
• Good understanding of LLMs, prompt engineering, and prompt-based interactions is a strong plus
• Excellent problem-solving skills and attention to detail
• Strong communication skills to collaborate with cross-functional AI/engineering teams

Good to Have

• Exposure to tools like LangChain, AutoGen, CrewAI, or similar agentic frameworks
• Experience with LLM evaluation frameworks or prompt testing tools
• Prior experience in AI/ML product testing or QA for conversational AI systems