Staff AI Engineer

• Architect, build, and operate production AI/ML systems and agentic workflows, including the tooling, infrastructure, and processes needed to develop, test, and deploy them reliably.
• Define and implement quality and reliability practices for AI systems — testing frameworks, monitoring, and pass/fail criteria that gate releases and catch regressions before they reach customers.
• Instrument and operate production systems, building sufficient visibility into system behavior (e.g., logging, tracing, dashboards) to diagnose issues and support continuous improvement.
• Translate production feedback and failures into concrete engineering improvements, driving iteration on models, prompts, retrieval, or other system components based on data and stakeholder input.
• Define and track the technical metrics that matter for a given system or product area (e.g., accuracy, latency, cost, reliability) and produce artifacts that product, engineering, and risk stakeholders can rely on.
• Establish technical standards, tooling, and patterns that can be reused across teams building AI-driven products, promoting consistency in how AI system quality is measured and maintained.
• Partner closely with engineering, product, and design; mentor other engineers; and serve as a technical resource on AI system design, quality, and reliability trade-offs.
• Translate ambiguous, evolving product and technical goals into concrete, measurable engineering plans and deliverables.

Education Minimum Job Requirements Bachelor's or Master's degree in Computer Science, Machine Learning, or a related technical field, or equivalent experience.

Certification/License None required.

Work Experience 5+ years building and operating production AI/ML or software systems, including hands-on experience with large language models (LLMs) and/or AI agent systems.

Knowledge, Skills & Abilities

• Strong software engineering fundamentals, including proficiency in Python and experience with distributed data pipelines and CI/CD.
• Demonstrated experience building and/or operating evaluation, testing, or observability capabilities for ML/AI systems in production.
• Experience with LLM-based system design, including techniques such as LLM-as-judge evaluation, golden-dataset curation, and regression/drift detection.
• Experience instrumenting or operating complex, multi-step systems (e.g., multi-step reasoning, tool use, retrieval, multi-agent handoffs).
• Experience assessing agentic or AI systems for security, reliability, or failure modes (e.g., red-teaming or similar practices).
• Ability to translate ambiguous product or technical goals into measurable, defensible engineering plans.
• Strong cross-functional collaboration and communication skills; able to represent technical trade-offs to non-technical stakeholders.

Preferred Job Requirements

• Experience with cloud AI/ML platforms (e.g., AWS Bedrock) and agent frameworks (e.g., LangGraph, Strands, or similar).
• Experience with cloud-native MLOps/LLMOps practices: packaging, CI/CD, A/B testing, and model or prompt versioning.
• Hands-on experience with evaluation or observability platforms (e.g., Langfuse, Braintrust, LangSmith, or similar).
• Track record of standing up a new technical capability (e.g., an evaluation or observability function) as a reusable, cross-team resource.
• Experience using synthetic data to increase test coverage for AI/ML systems.

Experience across multiple product domains or business contexts