Lead AI Test Automation Architect

Job Title: Lead AI Test Automation Architect

Location: Toronto, Canada (Remote)

Experience: 10–14 Years

Role Type: Architect / Senior SDET / AI QE Technical Lead

Role Overview

ITMC Systems is seeking an experienced Lead AI Test Automation Architect to drive the design, development, and optimization of next-generation, AI-powered quality engineering (QE) solutions and platforms.

This role combines deep technical expertise in test automation engineering with hands-on mastery of Large Language Models (LLMs), agentic AI frameworks, and enterprise-grade AI tooling. In this role, you will define overall quality engineering strategies, design scalable automation frameworks, mentor technical teams, and pioneer innovation across QE automation, AI agents, RAG pipelines, and MCP-enabled intelligent workflows.

Key Responsibilities & Required Skills

1. AI, LLMs & Agentic Systems

• Hands-on AI Expertise: Strong, direct experience working with LLMs, prompt engineering, RAG architectures, vector databases, and model evaluation techniques.
• AI Tooling & Frameworks: Proficiency with LangChain, Hugging Face, Transformers, and OpenAI/Ollama APIs.
• Agentic Frameworks: Practical experience with agentic frameworks such as LangGraph, AutoGen, or CrewAI.
• GenAI QE Solutions: Architect and enhance GenAI-powered quality engineering platforms, AI agents, and autonomous testing workflows.
• Context-Aware Automation: Implement Model Context Protocol (MCP)-driven, context-aware automation and intelligent CI/CD decision engines.

2. Automation Engineering & Test Architecture

• Core Development: Strong coding skills in Python, TypeScript, or Java.
• Framework Design: Architect and maintain enterprise-level automation frameworks across:
• UI Automation: Playwright, Selenium
• API Testing: PyTest, Requests, RestAssured
• Performance Testing: JMeter, Locust
• AI Asset Generation: Develop prompt-optimized, AI-generated test assets, test cases, and modern validation mechanisms.

3. ML/AI Engineering & Data Pipelines

• Machine Learning Libraries: Experience with PyTorch, TensorFlow, Scikit-Learn, and NLP/CV libraries (e.g., NLTK, BART, OpenCV).
• Data & RAG Pipelines: Build robust data/embedding pipelines and optimize retrieval mechanisms for RAG architectures.
• MLOps Integration: Implement end-to-end CI/CD for ML models, including model versioning, evaluation metrics, and retraining workflows.

4. Cloud, DevOps & Governance

• Cloud Platforms: Deep understanding of AWS, Azure, or GCP cloud architectures and their respective AI/ML services.
• CI/CD Integration: Seamlessly integrate test automation pipelines using GitHub Actions, Azure DevOps, or Jenkins.
• Security & Governance: Ensure all AI/automation infrastructure and environments are scalable, secure, and compliant with enterprise governance standards.

5. Leadership & Delivery Excellence

• Technical Mentorship: Provide technical leadership and mentor cross-functional teams on AI adoption and automation best practices.
• Stakeholder Alignment: Collaborate closely with developers, SMEs, and product management to align quality engineering solutions with business goals.
• Quality Strategy: Drive feature prioritization, end-to-end quality strategy, defect triage, and solution architecture.
• Full SDLC Engagement: Work across the entire Software Development Life Cycle (SDLC) in an Agile/Scrum environment, actively contributing to test design, execution, and analysis.