Senior Automation QA Engineer

Senior Automation QA Engineer

About the Role

We are looking for a Senior Automation QA Engineer to own end-to-end test automation across API and UI layers for our AI-native platform, including agent workflow validation.

Key Responsibilities

• Design and execute test strategies for API and UI layers using Pytest and Playwright (Angular).
• Own CI/CD quality gates in Azure DevOps, ensuring reliable build promotion and release readiness.
• Build and maintain coverage for agent workflow testing, including non-determinism, agent graph validation, and confidence score assertions.
• Develop reusable automation frameworks and data setup utilities backed by PostgreSQL.
• Contribute to LLM evaluation and observability workflows using Langfuse or similar tooling.

Required Qualifications

• 6+ years of experience in QA automation.
• Strong hands-on expertise with Pytest and Playwright, with proven depth in both API and UI automation.
• Solid proficiency in Python and experience working with PostgreSQL for test data and validation.
• Hands-on experience with Azure DevOps CI/CD pipelines and quality gates.
• Awareness of LLM/agent testing challenges (non-determinism, agent graph validation, confidence score assertions); exposure to Langfuse or LLM evaluation frameworks is a plus and can be built on the job.