AI Operations SME

Role Overview

We are looking for an experienced AI Operations SME / Agent Specialist to serve as a senior escalation point for complex AI and agent-related operational issues. This role focuses on diagnosing failures across prompts, context, workflows, tools, policies, and agent behavior, while translating recurring issues into actionable improvements for Operations, Product, and Engineering teams.

The ideal candidate combines strong hands-on experience with Generative AI and AI-enabled workflows, along with structured troubleshooting, root-cause analysis, and technical communication skills.

Key Responsibilities

• Serve as the escalation point for complex AI-related issues that cannot be resolved through standard operational playbooks
• Diagnose failures involving prompts, context, workflows, tools, policies, and repeated agent behavior
• Perform structured root-cause analysis of recurring AI workflow issues using available logs, context, and supporting information
• Differentiate AI/model-related failures from data gaps, policy gaps, workflow issues, and tool failures
• Reproduce issues and document findings clearly to support Product and Engineering investigations
• Develop and maintain advanced troubleshooting guides, knowledge articles, and operational playbooks
• Coach Operations teams on complex troubleshooting scenarios and appropriate escalation paths
• Identify recurring failure patterns and translate them into actionable feedback for Product, Engineering, and Operations teams
• Support prompt testing, regression testing, edge-case validation, AI evaluation, and defect reproduction as needed

Required Skills and Experience

• 5 to 8 years of experience in AI Operations, Automation, Product Operations, QA/Testing, Technical Support, or related fields
• Strong hands-on experience using Generative AI, including prompt engineering and context engineering
• Experience troubleshooting complex AI or automated workflows and performing structured root-cause analysis
• Understanding of AI agent behavior, including agent actions, tools, guardrails, retries, workflows, and escalation logic
• Demonstrated experience resolving complex workflow or system failures
• Experience creating troubleshooting documentation, knowledge articles, and operational playbooks
• Strong technical communication skills with the ability to collaborate effectively across Operations, Product, and Engineering teams
• AI testing experience is preferred and may become a larger part of the role as the program matures