Senior AI/ML Engineer

Job Title: Senior AI/ML Engineer

Location: Fully Remote Role

Duration: 12 - Months Contract with Possible Extension

Pay Range: $100- $102 Per Hour on W2

Job Brief:

Client’s AI/ML team is building and scaling the Product—the unified AI agent hub for tax and compliance—and the Avalara Avi Agent (AAA) orchestration backbone. We are seeking Senior AI/ML Engineers with deep hands-on expertise in LLM agent orchestration, Model Context Protocol (MCP), and evaluation pipelines. In this role, you will develop high-performance asynchronous orchestration services, integrate domain-specific MCP tool suites across Client's tax products, build evaluation benchmarks, and implement cohort-based gating to ensure reliable, audit-defensible execution in high-concurrency production environments.

Responsibilities:

- Agent Orchestration & Backend Development: Design, implement, and optimize multi-agent orchestration workflows (A2A), prompt templating, dynamic intent routing, and conversation memory using Python, asyncio, and FastAPI.

- MCP & Tool Integration: Standardize and integrate Model Context Protocol (MCP) servers and tools across Client’s products (AvaTax, Returns, Tax Research, ECM, Licensing).

- Evaluations & Benchmarking: Build automated evaluation pipelines (LLM-as-a-judge, deterministic assertions, harness evals) to benchmark agent intent accuracy, routing precision (HSR/FSR), and execution latency.

- Gating & Entitlement Routing: Implement dynamic cohort-based feature gating and IAM-DS entitlement evaluation to support controlled beta rollouts and multi-tenant access controls.

- Performance & Reliability: Optimize token consumption, response streaming, context caching, rate-limiting, and error handling for high-concurrency agent interactions.

- Production Readiness & Observability: Build telemetry pipelines, structured logging, and monitor dashboards across Datadog, Snowflake, and Open Telemetry to track model accuracy and system health.

Required Qualifications & Technical Skills

- Professional Experience: 5+ years of software engineering experience in Python with strong expertise in high-throughput asynchronous services (asyncio, FastAPI).

- LLM & Agent Systems: 2+ years of hands-on experience building production LLM applications, agentic workflows, function calling, tool use, and multi-agent interaction protocols (e.g., LangChain, Lang Graph, LlamaIndex, or custom frameworks).

- Protocols & Architectures: Solid understanding of Model Context Protocol (MCP), RESTful API design, Agent-to-Agent (A2A) orchestration, and Knowledge Graph/RAG patterns.

- Evaluation Methodologies: Demonstrated experience establishing automated LLM evaluation datasets, synthetic test benches, and scoring metrics for intent detection and tool selection.

- Cloud & Data Stack: Experience with AWS cloud infrastructure (ECS/EKS, Lambda, SQS/Kafka), vector databases, and enterprise data warehouses (Snowflake).

- Software Craftsmanship: Strong commitment to unit/integration testing, CI/CD automation, clear API documentation, and agile team collaboration.

Preferred Experience

• Prior experience working with enterprise SaaS, tax/compliance technology, or complex multi-step workflow automation.

• Hands-on experience with streaming architecture, semantic caching, and LLM latency optimization.

• Familiarity with IAM/RBAC entitlement models and enterprise data security standards (SOC 2, ISO).