A
Anaplan
QA Automation Engineer (Artificial Intelligence)
Skills
PythonTesting & QAPostmanMachine LearningAI
15 days ago
Share this opportunity
Share this opportunity to other talents of your network:
✓ Offer them a visibility boost with clients.
✓ Help your contacts find their next job.
Important information
Contract type:
Permanent
Location:
London, UK
Work mode:
Onsite
Published on:
7 October 2026
What they need
- We’re pioneering a new role focused exclusively on quality assurance for GenAI/Agentic systems. As our QA Automation Engineer within AI, you’ll develop testing strategies, evaluation frameworks, and quality metrics specifically designed for LLM-powered applications. This role requires a unique blend of QA expertise, understanding of GenAI behaviour, and automation skills to ensure our AI features are reliable, accurate, and trustworthy
- Develop and execute end-to-end automated test coverage across evaluations, APIs, UIs, and performance tests using Python
- Implement testing strategies for GenAI features, including conversational AI, agentic systems, and LLM-powered workflows within your workstream
- Run and maintain automated test suites for prompt testing, including regression tests designed to detect unintended changes in model behaviour
- Support and maintain evaluation frameworks to measure GenAI performance across accuracy, relevance, safety, consistency, and latency
- Contribute to curating high-quality test datasets and “golden examples” that represent diverse user scenarios and edge cases
- Assist in adversarial testing to identify potential model hallucinations or biases, and support the integration of production alerting systems
- Utilise internal tools and testing frameworks to help developers easily validate their GenAI code, and assist with user acceptance testing
- Take accountability for your workstream’s automated coverage, documenting testing procedures and known issues in clear, accessible formats- Familiarity with CI/CD pipelines and automated testing integration
- Solid QA or test engineering experience, with exposure to or a keen interest in AI/ML systems
- Experience with API testing tools (Postman, REST Assured) and backend testing
- Hands-on Python coding experience
- Ability to execute test cases and evaluation criteria for non-deterministic systems
- Good understanding of GenAI technologies including LLMs, prompt engineering, and AI application patterns
- Practical knowledge of software testing methodologies (functional, integration, regression, performance, security testing)
- Solid experience with test automation frameworks and scripting (Python, Pytest, API, and UI Testing)
Other offers great for you!
These companies are also looking for great profiles
i
iSupplier
Test Management / Quality Assurance / Test Automation
Freelance
Brussels, Belgium
Hybrid
Skills
Testing & QATest EngineerQA Engineer
6 months ago
Quick apply
T
Tenth Revolution Group
QA Engineer
Freelance
Ghent, Belgium
Hybrid
Skills
QA Engineer
2 months ago
Quick apply
L
LeHibou
QA Tester
400€ / jour
Freelance
93400 Saint-Ouen-sur-Seine, France
Hybrid
Skills
Testing & QA
6 hours ago
Quick apply
No company information available.