AI QA Automation Engineer

Key Skills: Automation Testing, Selenium

Roles and Responsibilities:

• Design, develop, and maintain automated test frameworks for AI-powered web applications, APIs, and backend services.
• Build automated test suites for LLM-based and agentic AI platforms, including functional, integration, regression, and end-to-end testing.
• Develop test strategies for non-deterministic AI systems using statistical validation, semantic similarity, and confidence-based assertions.
• Perform failure injection, resilience, and recovery testing for asynchronous and event-driven workflows.
• Validate multi-tenant security, data isolation, sandboxing, and access control through automated testing.
• Automate API testing, workflow validation, and release verification within CI/CD pipelines.
• Develop synthetic test data and reusable test harnesses to improve automation coverage and reliability.
• Execute performance, scalability, concurrency, and reliability testing for AI platforms.
• Collaborate with AI engineers, platform engineers, and product teams to define acceptance criteria and improve system quality.
• Mentor QA engineers and contribute to reusable automation frameworks and best practices.

Skills Required:

• 8+ years of experience in QA Automation, API Testing, AI/LLM testing, and automation framework development.
• Strong experience in Automation Testing and QA framework development.
• Hands-on expertise with Playwright, TypeScript, and Selenium.
• Experience with API Testing, backend validation, and integration testing.
• Strong understanding of AI/LLM testing, prompt validation, RAG systems, and non-deterministic application testing.
• Experience testing asynchronous and event-driven systems.
• Knowledge of CI/CD tools such as GitHub Actions, Jenkins, or Azure DevOps.
• Strong SQL skills and experience with data validation and synthetic test data generation.
• Experience with test automation frameworks, release gates, and quality metrics.
• Familiarity with security testing, adversarial testing, and OWASP LLM Top 10 concepts.
• Strong analytical, troubleshooting, and communication skills.

Good to Have:

• Experience with Temporal workflows or similar orchestration platforms.
• Knowledge of performance testing tools such as k6 or Locust.
• Familiarity with LangSmith, Langfuse, Promptfoo, DeepEval, or Arize Phoenix.
• Experience with RAG, OpenTelemetry, OCR, or document processing systems.
• Exposure to SOC 2, GDPR, accessibility testing, or qTest.
• Experience testing multi-tenant SaaS platforms and AI agent frameworks.

Education: Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.