Quality Engineering Lead

Quality Engineering Lead

Blue Machines Full-time

About Blue Machines

Blue Machines builds the platform for production AI agents - agent builder, real-time voice agents, workflow orchestration, integrations, and inference. Multi-tenant systems, non-deterministic behaviour, many providers in the call path, and high bar for what ships to customers.

About the role

We are hiring a founding Quality Engineering Lead - a builder who designs and ships the systems that decide what is safe to release. Your output is platforms and mechanisms, not test cases. You may grow a team later; you are hired to build first.

What you will do

• Design and build the test platform: execution, isolation, test-data lifecycle, reporting, flake handling, and a clear answer to whether a build should ship
• Build an agent evaluation harness that detects behavioural regressions after prompt, model, tool, or config changes
• Own release gating: enforceable gates in code, post-deploy verification, canary criteria, and a narrow hotfix path
• Treat capacity and load as release criteria, not optional reports
• Turn customer-visible defects into durable detection (owned, staged, measurable)
• Push testability into product systems - seams, replay, fault injection, observability
• Raise the engineering bar through review and mechanisms, not by writing every test yourself

What we are looking for

• Strong software engineering background - you can design a system from an ambiguous problem and implement the hard path
• Experience owning quality, reliability, release, or evaluation platforms (not only writing suites)
• Experience with conversational AI, voice, or LLM evaluation - or a clear track record evaluating non-deterministic systems without ground truth
• Release engineering, progressive delivery, or SRE depth - gates, canaries, and production signal you have owned
• Hands-on building of CI/CD quality infrastructure at meaningful scale
• Product mindset - you tie quality work to customer-visible risk and delivery outcomes
• Evidence over assertion - you quantify, and you separate known from assumed
• Ability to work with a small team and influence engineers who do not report to you

This role is not

A manual QA role, a checklist sign-off role, or a hire to write Playwright/pytest cases as the main job. We already have deep automation capability; we need the architect who builds what sits above it.

Disclaimer: This job description has been sourced from a public domain and may have been modified by Naukri.com to improve clarity for our users. We encourage job seekers to verify all details directly with the employer via their official channels before applying.