AI Automation Engineer - Urgent

Company Overview

NCS is a leading AI Tech Services company. With a 15,000-strong team across the Asia Pacific, NCS scales its platforms and capabilities to provide clients with greater agility and AI expertise across a range of industries. Embracing a strong ecosystem of global partners, NCS transforms technology services delivery combining AI with digital resilience to drive real business impact. NCS is a subsidiary of the Singtel Group.

An AI Tester / AI Automation Engineer evaluates and validates AI models and workflows to ensure they are accurate, safe, and reliable. They design tests for both traditional software constraints and non-deterministic AI outputs, checking for hallucinations, bias, and performance. This role bridges standard quality assurance and machine learning operations.

Key Responsibilities

• AI Evaluation: Build evaluation pipelines to assess model behavior, factuality, toxicity, and task success.
• Test Automation: Develop and maintain automated test scripts using Python and frameworks like Pytest.
• System Monitoring: Continuously monitor AI models in production environments for quality degradation and system drift.
• Bug Identification & Red Teaming: Find creative ways to break AI products, including adversarial stress testing, jailbreaks, and prompt injections.
• Cross-Functional Collaboration: Work closely with data scientists, product managers, and developers to improve prompt design and workflow structure.

Required Qualifications & Skills

• w/ experience either of the ff:

  • playwright
  • AI Testing
  • Python Based
  • Karate Testing

• Experience: Typically 3+ years in QA or test automation, with at least 1-2 years directly focused on AI/ML-enabled systems.

• Programming: Strong scripting skills, heavily preferring Python.

• Automation Tools: Proficiency in UI and workflow test automation using tools like Playwright or Selenium.

• AI Tooling: Familiarity with LLM evaluation and observability frameworks like DeepEval, Ragas, or Evidently.AI.

• Analytical Mindset: Excellent debugging skills with an eye for spotting data anomalies and edge-case failures