Senior GenAI Quality Engineer and Solution Analyst

Key Responsibilities:

• Define and execute end to end test strategies covering UI workflows, backend APIs, integrations and agentic interfaces.

• Test conversational and agentic behaviour including multi turn context, tool selection, tool inputs and outputs, state transitions, retries, timeouts, handoffs, approvals and recovery from partial failure.

• Validate GenAI responses for task completion, grounding, relevance, consistency, citation behaviour and safe failure, while recognising that outputs can be non deterministic.

• Perform functional, integration, regression, exploratory, negative, resilience and basic performance testing across application layers.

• Design API tests for contracts, authentication, authorisation, validation, error handling, idempotency, rate limits and downstream failures.

• Test UI behaviour across browsers and realistic user journeys, including loading states, interrupted sessions, feedback capture, accessibility basics and clear error communication.

• Create and maintain test data, reusable test scenarios and traceable evidence suitable for enterprise release governance.

• Use logs, traces, request and response payloads and observability tools to isolate defects and distinguish application, model, data, integration and platform issues.

• Automate the tests that materially reduce cycle time, manual effort or production risk, and keep unstable or low value scenarios out of the automation suite.

• Communicate defects and quality risks clearly to engineers, product owners, GenAI specialists, security teams and business stakeholders.

• Provide an evidence based release recommendation, including known limitations, residual risks and areas requiring monitoring.

• Partner with product owners, business users, architects, engineers and GenAI specialists to define the problem, target user journeys and expected business outcomes.

• Analyse proposed GenAI use cases and determine where deterministic application logic, retrieval, workflow orchestration, tool using agents or human approval should be used.

• Translate business requirements into end to end solution flows, functional requirements, interface behaviours, decision rules, acceptance criteria and non functional requirements.

• Map interactions across user interfaces, APIs, models, prompts, retrieval components, enterprise data sources, agent tools and downstream systems.

• Analyse solution options and document tradeoffs relating to quality, complexity, cost, latency, security, data access, maintainability and operational risk.

• Identify unclear ownership, missing controls, integration assumptions, failure scenarios and operational gaps before development begins.

• Support the design of human approval, fallback, escalation and exception handling paths for agentic solutions.

Skills

• UI and API Quality Engineering
• Agentic and GenAI Application Testing
• Risk Based Test Automation and Observability