Sr. Generative AI Tester (Airline Domain)

• Develop and support AI applications using Python, Generative AI, and Large Language Models (LLMs).
• Design, execute, and automate comprehensive AI test strategies across the AI application lifecycle.
• Validate AI models, LLMs, AI agents, and agentic workflows for accuracy, consistency, reliability, and safety.
• Perform functional, integration, regression, performance, security, and user acceptance testing for AI solutions.
• Conduct LLM evaluation, prompt testing, hallucination testing, bias and fairness assessments, and guardrail validation.
• Develop automated frameworks for testing AI agents, chatbots, copilots, and autonomous workflows.
• Define and track AI quality metrics, benchmarks, and acceptance criteria.
• Identify, analyze, and resolve defects related to AI outputs, workflows, and model behavior.
• Collaborate with business stakeholders and development teams to ensure production-ready AI solutions for airline operations.

Desired Candidate Profile

• Strong programming expertise in Python.
• Hands-on experience with LLMs, Generative AI, RAG architectures, Vector Databases, and AI Models.
• Experience with AI frameworks such as LangChain, LangGraph, CrewAI, AutoGen, LlamaIndex, or Semantic Kernel.
• Experience developing and testing AI Agents, Agentic AI solutions, Chatbots, and Conversational AI platforms.
• Strong expertise in AI Testing, including:
• LLM Evaluation & Validation
• AI Agent Testing
• Prompt Testing & Optimization
• Hallucination Testing
• Bias & Fairness Testing
• Functional Testing
• Integration Testing
• Regression Testing
• Performance & Load Testing
• Security & Responsible AI Testing
• Test Automation for AI Applications
• Experience creating AI quality frameworks, test cases, evaluation datasets, and benchmarking methodologies.
• Familiarity with Azure OpenAI, AWS AI, Google AI services, or equivalent platforms.

Domain Experience

• Experience in the Airline, Aviation, Travel, or Transportation domain is required.
• Knowledge of airline operations, reservations, customer service, crew management, airport operations, or passenger experience solutions is an advantage.

Preferred Qualifications

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Data Science, Engineering, or related field.
• Certifications in AI, Machine Learning, Software Testing, or Cloud Technologies are preferred.